Can I copy my Ubuntu OS off my hard drive to a USB stick and boot from that stick with all my programs?

I've been doing this for years. I'm writing now from a PC which does not even contain an internal hard drive. I don't even carry a laptop, just this high performance USB flash media.

I will now outline the two components you need to buy if you want decent performance. Most flash drives are too slow for most people, and they can also become unstable due to overuse of the type they get running operating systems and particularly swapspace.

I use this USB3 to mSATA SSD flash media adapter, priced here at US$19.

And inside it, I use an mSATA SSD drive originally made for today's tablet and netbook computers. Though I manage to get by on just 64 GB myself, you may choose to buy something with greater capacity. Last I checked, the capacity of these drives was up to around 1 TB. I remember when the largest available was just 32 GB.

If you would like something still more rugged, I can recommend this similar USB3-mSATA adapter which is made from a solid section of extruded aluminum. It is almost exactly the same price. Whereas the model I previously recommended is only 2/3 the thickness, it would probably cave if stepped on. This one might tolerate a car driving over it.

To the best of my knowledge, nobody has made a waterproof model yet, but it's only a matter of time. The strange thing is that as obvious as these appear as portable operating system environments, apparently all references to these regard their use entirely for portable data.

One last detail. It is my practice to keep a 128 GB common slow flash drive in one of the more regularly used computers and have the portable drive automatically back up critical data to it when it is used there. The automatic backup software I'm using is that which is included with Ubuntu and it encrypts its backups. This way whether I lose the boot drive, or my home is robbed, the odds of actually losing all my current personal data are incredibly low.