Crowds

While looking around for inspiration, I came across a photo on the Internet of a busy street in New York. The street was filled with people, cars, and buildings - three of my favourite subjects. The cars and buildings were pretty straightforward, but how was I going to render hundreds of people?
I've rendered scenes in the past with multiple people. The main problem is that the character meshes and textures consume a great deal of memory (up to a few hundred megabytes each!). I came up with some tips for reducing the memory consumption for my Ipod in D minor image. Here, I needed a different method - one that would allow me to insert hundreds of people. What about a two dimensional solution?


Billboards A billboard is a 2D image that is inserted into a 3D scene. Using DAZ\|Studio, I created a number of 2D images with people (many were models from previous projects). After creating a character, I used the "Front View" (orthographic) camera and positioned the person such that the shoes were touching the bottom of the image. This positioning allows the billboard's shadows to look correct. Next, I rendered the character from eight different angles (45 degrees apart). I used DAZ\|Studio's timeline (animation tool) and rotated the model over eight frames. Later, I created two sets of billboards per character: one with flat lighting and one with side lighting. The lit billboards allowed for a bit more realism.
	I set my billboards to 1000x1000 pixels - probably larger than I needed. I set the renderer to output each frame as a separate numbered image (see dialog box at left). I saved the images in .PNG format which supports an alpha channel. This lets the "background" show through the billboard. Note: DAZ\|Studio appends the frame number at the end of the each output filename. The filenames I picked for each billboards contained the pose (walking or standing), the character number and the angle the character was facing. This scheme is used by the crowds macro to select the appropriate billboard.
All in all I created about 70 characters (most were from previous projects). About 10 characters were standing and 60 were walking. For each character there were 8 billboards (with flat lighting) and 8 billboards with side lighting. As of October 2008, I had over 1100 billboards. The macro to draw the billboards allowed me to select the character, the pose (standing or walking), and the direction the character was facing. These three parameters are used to build a filename that selects the specific billboard image. Additional parameters include the scale (how large to make the billboard), lighting (flat lighting, left lighting, right lighting), the position to place the billboard, and the camera position. The camera position is important because the billboard must be rotated to be perpendicular to the camera.
I arbitrarily decided to render each of the billboards at 1000x1000 pixels (250-750kB). Obviously for distant crowds, they could be much smaller. With many instances of these billboards, my scenes were consuming lots of memory. For example, the sample scene at right has 2000 billboards (of only eight characters). This scene consumed 840 MB. I found that POV-Ray would reallocate memory for every billboard instance, even the duplicates. So I rewrote the crowd macro to only allocate each image only once. After this change, the same scene consumed only 33 MB. The key was keeping track of each billboard image as it was needed. Then checking to see if it was already allocated. I wasn't sure that POV-Ray would allow me to create an array of objects, but sure enough it worked. Here's the magic code:

Some examples of scenes with my crowd macro. The audience in my revision of Ipod in D minor.

A redo of the parking lot scene. This scene added 3473 billboards to the original scene (with only two people). Although the new scene consumed an additional 2 GB, it rendered 3 seconds faster!

Autumn in NYC.

A crowd of people standing in the form of the POV-Ray logo.

Note in this image how the people in the nearest three lanes have flat lighting (they're in the shade), while everybody else is lit from the side.

Here's a description of the advantages and disadvantages of using 3d models vs. 2d billboards.

	3d model (mesh2 format)	2d billboard
memory requirements	typically 100s of MB (with high resolution textures) per model limited to <20 models per scene on most computers	depends on billboard size - most of my 1000x1000 pixel billboards are about 150-450 kB can render hundreds or thousands of people in a scene
instancing (multiple copies)	additional (mesh2) copies require very little additional memory additional instances can be randomly scaled and rotated to make them look unique	duplicate billboards can be instanced such that the memory for the image is only allocated once duplicate billboards are perfect clones of one another
interaction with environment	accurate lighting, shadows, reflections, and media	inaccurate lighting, shadows, and reflections media artifacts on transparent parts of billboard overlapping billboards may require a higher max_trace_level settings to avoid artifacts
placement	object will render correctly from any camera angle can be transformed/rotated easily	billboard must be rotated perpendicular to camera, cannot arbitrarily transform the billboard if camera is moved, the billboard must be rotated again and may no longer be facing the same direction as intended camera cannot be too high, since the "top" of the billboard doesn't exist
resolution of detail	renders well for closeups or distant views can use high resolution textures (as memory permits) to get extreme closeups	closeups possible depending on the resolution of billboard image some perspective distortion may be visible in closeups (especially if the billboard is rendered with an orthographic projection)
render time	slow (especially hair with transparency)	very fast - although overlapping billboards may require a higher max_trace_level setting (leading to longer renders) to avoid artifacts

Here's a comparison of the billboard vs. a real 3D model. For some reason, the textures didn't convert properly, so the colours are slightly different.

Note in the orthographic projection, one (or both) of the person's feet always touch the bottom of the billboard. This was important because the bottom of the billboard touches the ground and we don't want the people to be floating off the ground. Once the figure was properly positioned in DAZ Studio, a simple rotation of the person did not change the "height" of her feet.

The perspective projection is more accurate as can be quickly seen by looking at the feet. The image appears slightly more three dimensional. When rendering this billboard, I positioned the camera at roughly the same height as the person's head. This point of view means that we will see more of the top of the feet and bag. Unfortunately, when rendering the perspective projection, the "height" of the feet changed everytime I rotated the figure. I had to manually crop the billboard to keep the person grounded.

The 3D model is similar to the perspective billboard, but has more accurate lighting and shadows. Note: her glasses reflect the environment and her pants cast shadows on her legs. Unfortunately replacing the billboard with a 3D model added about 300MB to the scene.

In conclusion, 3d models result in more accurate renders, but come at a large cost in memory and rendering time. The billboards are less accurate, but consume much less memory and rendering time. In general, billboards are probably a better solution for distant figures where shadows, reflections, artifacts, and perspective distortions are not visible. In addition, a large number of billboards are required to avoid the army of clones problem.

My crowd macro tries to overcome many of the limitations:

When a billboards is first defined (and memory for its image allocated), the billboard is stored as an object. If the same billboard is used later, the macro will use the previoiusly defined object, thus not re-allocating the memory for the image. This techniques allows for scenes with thousands of billboards
To improve the interaction with scene lighting, I created two sets of billboards for each character: one with flat lighting - the other with side subtle lighting. This technique allowed me to place billboards in both light and shade and have a semi-realistic lighting. The lit billboards are lit from the left. To get right-side lighting, I mirror the billboard.
When flat lighting is used, some of the billboards are randomly mirrored to increase the variation of the crowd.

Click here to download the crowd macro and a sample scene. There are eight characters (four women; four men).