Normals are an important piece of geometry for rendering smooth surfaces. Generation of proper normals is very straightforward for continuous surfaces. To calculate a smooth normal for a vertex, average the facet normals of all the polygons that the vertex is in. However, this does not work when there are edges that are to be preserved in the model. This method will smooth out all edges. Therefore, it is desirable to selectively average normals according to a threshold for edge preservation. 
A good example is a model of a cube. It is desirable for the cube to have flat faces. However, if all the facet normals around a vertex are averaged, and the resulting average normal is used, the cube takes on a spherical look (see figure 1), which is probably not the desired effect. 
Another gotcha when creating smooth normals is duplicate vertices. These can be eliminated by 'welding' vertices that are closer than a tolerance to each other. Then the normals are recalculated. See figure 2 for an example of this. 
Different models require different edge tolerance angles. Sometimes the angle across some edges is the same as the angle between some geometry that should be smooth. This is a problem with my current algorithm. It could be dealt with by localizing the averaging according to user defined areas. See figure 3 for an example of problems of this kind (the base of the pawn is faceted, when it shouldn't be, but the tolerance must be 76° for the edge in the middle to appear). 
/* glmVertexNormals: Generates smooth vertex normals for a model. * First builds a list of all the triangles each vertex is in. Then * loops through each vertex in the the list averaging all the facet * normals of the triangles each vertex is in. Finally, sets the * normal index in the triangle for the vertex to the generated smooth * normal. If the dot product of a facet normal and the facet normal * associated with the first triangle in the list of triangles the * current vertex is in is greater than the cosine of the angle * parameter to the function, that facet normal is not added into the * average normal calculation and the corresponding vertex is given * the facet normal. This tends to preserve hard edges. The angle to * use depends on the model, but 90 degrees is usually a good start. * * model  initialized GLMmodel structure * angle  maximum angle (in degrees) to smooth across */ 
Smooth normals for a cube generated with a 91° edge tolerance (smooth across the edges). Eight normals generated (one per vertex).  Smooth normals for a cube generated with an 89° edge tolerance (not smooth across the edges). Thirtytwo normals generated. 

Flat shaded teapot (one normal per face).  Smooth normals generated (with a 90° edge tolerance). The lines are where the bezier patches overlap. There are duplicate vertices along the lines. 
Optimized teapot model (duplicate vertices removed).  Wireframe teapot to illustrate structure. 

Flat shaded pawn.  Smooth normals generated (with a 90° edge tolerance). Notice the sharp edges in the model. 
The third pawn image with the good specular highlight has normals that were created with an edge tolerance of 76°. These edge tolerance angles are very model specific.  
Smooth shaded pawn. 

Flat shaded head.  Smooth normals generated (with a 90° edge tolerance). Notice the sharp edges (especially around the eyes) in the model. 
