The Must Haves
At the very minimum, you'll need a good sharpener, eraser and a few sheets of paper.
You don't need an expensive sharpener for your pencils, anything with a sharp blade will get the job done. But some pencils (looking at you Prismacolor) can break very easily, so you might want to get a suitable sharpener. Or you might have a few oddly sized pencils that won't fit into a regular sharpener. Some people swear by electric and others will tell you that manual is the way to go. What I have found is that certain types of sharpeners work best with certain pencils and finding out which is the best fit for your pencils can be a trial and error process.
I personally prefer manual sharpness because then I don't have to worry about changing batteries or charging yet another device in my home. My favorite watercolor pencils will not fit into most regular sharpeners so I ended up buying the Dahle 133. It produces a nice long point (you can even change the length/sharpness of the point) and has an over-sized hole that can take pencils of any shape and size. The KUM brand of sharpness also have pretty good reviews with most pencils and they are very cheap to boot.
Erasers also come in many variants - electric, kneaded, pencil type erasers and so on. Electric erasers can be very handy for tiny spaces while kneaded erasers are useful for lifting off excess pigment. Any good quality eraser will work for colored pencils although it may not always be possible to completely erase your marks. Just be careful not to rub too hard – you might end up erasing the print or even tear the paper. I use this Tombow eraser which is very useful for erasing small details and spaces.
Sheets of Paper
You don't need any kind of special paper but just a few sheets of your average printer paper. I always keep a couple of sheets behind the page I am coloring to prevent bleed through or the ink transferring to the opposite page. It also helps prevent the indentations from the ruining the next picture. After I have completed an image, I place a sheet of paper in between as well so that any excess pigment is not smeared on the opposite side.
The Nice to Have
Basically that's all you need to get started - a book or a few sheets of artwork, medium (pencils or pens), a good sharpener and an eraser and a few sheets of paper to place in between. But there are many other things that can enhance your coloring experience or the end results. None of these are essential to getting started and so I would recommend getting them as and when you feel the need.
As you begin to use more advanced coloring techniques such as shading, layering and blending, you might feel the need to get some special tools that help get a smoother finish on your artwork. Blending pencils and markers are available which can enhance the appearance of your colors and seamlessly blend the transitions from one color to the next.
Blending pencils are available from Caran D'ache, Prismacolor and Derwent to name a few. Colorless markers achieve the same thing for alcohol or water-based pens. Many people also use tortillions or paper stumps to blend color pencil pigment. Many artists use odorless mineral spirits (such as Gamsol) or even Vaseline to blend color pencils. Although I haven't tried any of these techniques (I only use a blender pencil), the end results can be exquisite.
Although sharpeners work very well for pencils, you can end up wasting pigment. Sometimes the pencil tips can become dull even though there is plenty of lead exposed and the pencil doesn't actually need sharpening. In such situations, a sandpaper block can be very useful. You can just rub the pencil across to get a sharp tip.
Although not essential, soft pastels can be very useful for covering large areas such as the sky in landscapes. Quite a few artists color everything on a page with pencils but it can take up a lot of time if you want good results. Soft pastels on the other hand, give excellent results with minimal effort. You can get a high quality set of pastels from Faber Castell for just ten dollars, and even the smallest 12 pack should be sufficient for colorists. Fewer colors but higher quality pigment will get you even coverage rather than splurging on big sets and struggling to get any color to spread on the page.
And there you have it. Everything on this list can be bought as you feel the need for it. You don't have get everything before starting to color. The important thing is to get coloring!