When you no longer want an app on your iPod, you can uninstall it by pressing on the app’s icon until it starts to shake. As it is shaking, you will notice an “X” in the upper left hand corner of the app. Just touch that “X” and choose “Delete”. This will remove the app from your iPod. You will then need to make sure the deleted app doesn’t sync back onto your iPod during the next sync. To do this, plug your iPod into your computer and open iTunes. In iTunes, go to iPod>Applications and deselect the app from your Sync Applications list.
There is no way to delete factory installed apps from your iPod, but you can hide the icons by enabling parental controls through Settings>General>Restrictions. Choosing which apps you want to have restricted will then hide the icons.
Resource: About Apple