The sense of taste begins with the taste buds, located on top of the fungiform papillae, or the large bumps on the tongue. Other taste receptor cells can be found on the palate and in the throat, but the tongue has the most. The fungiform papillae are shaped similar to mushrooms and sometimes swell a little when stimulated. Alongside the fungiform papillae are the filiform papillae, little brush-shaped protrusions that usually lack receptor cells.
Contrary to what you may have heard, the tongue is not divided into taste sections. This is a myth based on a mistranslation of a German book which has been perpetuated in schools since the early 1900s. If you put a tiny bit of salt or sugar on different sections of your tongue, you'll see that you can taste it anywhere.
The five acknowledged tastes are sweet, sour, salty, bitter, and umami. Umami could just as easily be called savory, but it is named after the Japanese word for the same thing because a Japanese researcher first discovered it in 1908. This discovery co-occurred with the chemical isolation and subsequent marketing of MSG as a flavor enhancer.
Just as important to the sensation of this sense are the olfactory cells in the nose which detect scent. What we perceive as taste is a complex interplay of smelling and tongue-tasting. The nose, tongue, eyes, and brain all evolved together to ensure that we consume the good stuff and keep out the bad stuff: rotten foods, poison foods, and other indigestibles.
An important and often unmentioned component of this sense is the gustatory cortex, a section of the surface of the brain near the back, which processes taste inputs. It is located next to the parts of the brain which control chewing and swallowing. About 25% of the population are "supertasters", experiencing a heightened sense, partially due to a greater density of taste buds and partially due to subtle brain differences in how this sense is processed.