There are many books on the basic physics of optical systems, and the design of optical systems for physics and chemistry applications. The following books are particularly clear and practical:
The Hecht book is a general reference for beginners. The first Welford book covers classical optics: everything you might expect to find in a physics textbook. The second Welford book covers additional topics that one must understand to assemble real lens systems, such as the materials lenses are made from, aberrations, geometrical distortion, and the cos^4 law.
Much of the material in classical optics books is only accurate for narrow angle optical systems. In optics jargon, the rays are assumed to be "paraxial". Remember: the primary users of these books are using the optics to guide laser beams, design microscope objectives, and the like. Furthermore, the simple pinhole perspective model is only guaranteed to be reasonably accurate for single-element narrow-angle lenses.
Photographic lenses, are almost invariably multi-element designs and frequently have moderate to wide fields of view. Analysis of these lenses is more complicated and largely done documented in books on photographic optics.
A very clear introduction to photography and photographic optics, which everyone should own:
A more comprehensive, detailed, and technical survey of photographic optics:
Slightly more technical books on lens design:
If you'd like to understand the non-optical features of your 35mm camera, such as the film holder, the focus aids, the exposure meter, and the SLR viewfinder, see