Bitcoins are 'stored' in something called bitcoin addresses - they look something like this:
There are two parts to a bitcoin address - the public key (commonly just called the address) and the private key - the important part that lets you spend the bitcoins on it's corresponding public key. Bitcoin is an example of public key encryption, as you can give out the public key freely but you must keep the private key to yourself.
To send bitcoins to an address, a message is broadcast from the owner of the sending address to the network that X amount of coins from that address now belong to the new address. This operation is authorized by the sender's private key, and if he doesn't have the private key he can't spend the coins, plain and simple.
BItcoin addresses are created by first picking a random number (for the all important key) and creating an ECDSA (Elliptic Curve Digital Signature Algorithm) public/private key pair with them. This operation alone generates the private key - but Bitcoin addresses are not simply public keys, but rather modified versions of them. The generated public key is then put through several SHA-256 and RIPEMD-160 operations, until eventually being converted into a format called Base-58. Base 58 is an encoding that removes the possibility of similar looking characters, such as lowercase L and uppcase I, as well as 0 and O. Finally an identifying number is added to the beginning of the address - for most bitcoin addresses, this is generally 1, indicating it is a public bitcoin network address.