A couple of months ago, the batteries on my Honeywell wireless smoke detector were low and I was looking for inexpensive replacements. The batteries are used in other Honeywell sensors as well. The best deal I could find on Ebay were Panasonics shipped from Germany. I checked Walmart in the US but the Rayovacs I had found previously were not in stock and no longer on their website.
While looking for 9V alkalines at Dollarama (fairly good deal at $2.50 each), I spotted the Panasonic CR123A batteries for $3. Dollarama is also a good source for CR1032 button cells. Not all stores may carry them so you may need to search a few different ones.
Update: The CR123A seem to be no longer available at Dollarama. I ordered some on Amazon.ca from a third party seller for around $8 for two pieces including shipping (around 2 weeks for delivery). MEC also carries them for about $6 each and Princess Auto has Westinghouse brand for around $6
Keywords: cheap, inexpensive, CR123, CR123A, battery, batteries