Apr 1, 2020 4:58 PM

A Brief History of the Census—and How Covid-19 Could Change It

Census-taking used to spur innovation. Now, the pace of technology, and the challenge of going door-to-door, could force it to adopt existing systems instead.

The United States is currently undertaking its population census, the once-a-decade effort to count every single person living here. More than 150 other countries will join in this world census round—the 10-year period centered on 2020, and around 90 percent of the world’s population has already been or will be counted.

But while impressive on a human scale, a modern census is technologically unspectacular. The American questionnaire asks just seven questions of each person, with a few more for each household as a whole. Counting generously, the government is collecting 130 bytes of data per person and perhaps another 13 bytes per household—amounting to around 45 GB for the whole US population. If it weren’t for the strict confidentiality, you could, once it’s complete, carry around the entire contents of the US census on a base model iPhone 11. The challenges of this year’s census are enormous, but they are social, political, logistical—and now public-health related—rather than strictly technological or scientific.

Adapted from The Sum of the People: How the Census Has Shaped Nations, From the Ancient World to the Modern Age by Andrew Whitby. Buy on Amazon.

Courtesy of Basic Books

That wasn’t always the case: For centuries, counting its people was amongst the most technologically complex procedures a state could undertake. It’s not possible to point to a first census: Such a simple idea likely arose independently many times and in many places. Although writing is now integral to how we record, analyze, and report data, census-taking certainly predated it. Herodotus relates a story of the Scythians, nomadic warriors who lived in Central Asia in the first millennium BCE. “Because their king, who went by the name of Ariantas, wished to know how many Scythians there were, he gave orders that each one of them was to bring an arrowhead ... A huge number of arrowheads were duly brought, and the king”—presumably having first counted them—“decided to fashion a monument out of them that he could bequeath to posterity.” Such a method would be a bit unwieldy today, but it probably worked fine for the Scythians.

In fact, proto-literate societies could conduct censuses of great complexity. The Inca, who dominated Andean America at the time of the Spanish conquest in the 15th century CE, recorded elaborate census statistics in connection with taxation, despite lacking what we would recognize as writing. Instead they used knots tied in complex bundles of colored strings called khipu to encode information. Of the thousand khipu that exist in museums today, two-thirds contain numerical data, and at least 50 are thought to pertain to censuses.

The most important ancient census, from our modern perspective, was surely that of Rome. Romans adopted the census late in the city’s regal period, shortly before the more familiar republic was declared in 509 BCE. They gave the procedure the name we now use (from the Latin cēnsēre, “to assess”), made it fundamental to their social order, and spread it yet further as their territory grew.

The censors, the officeholders who conducted the census, determined each Roman citizen’s position in an elaborate class hierarchy. This, in turn, dictated how that citizen and his family lived: how he could dress, how the law treated him, and how he could exercise political power. In principle, this stratification of Roman society was performed every five years. Each head of household would be called upon, in turn, to make his declaration. He would give his full name, that of his father—or patron, in the case of a freed slave—and his age. He would report his marital status and, if applicable, the name of his wife and the number, names, and ages of his children. He would then move on to an account of his property. This would proceed by tribe until it was complete. The censors were appointed for an 18-month term, and sheer logistics suggest the count must have taken most of this time.

The census survived the end of the Roman Republic and the transition to empire, though it receded in importance. Caesar Augustus considered three censuses, in 28 BCE, 8 BCE, and 14 CE, amongst his life’s achievements. The last recorded a count of 4,937,000 people, a number etched upon stone throughout the empire. In the biblical account of Luke 2:1, Mary and Joseph were called to travel from Nazareth to Bethlehem, because “Caesar Augustus issued a decree, that a census should be taken of the entire Roman world.”

Unlike today’s censuses, ancient censuses rarely counted everybody, often including only, for example, men of fighting age. So while they dominate the theological account, Mary and the infant Jesus would probably have been excluded from the administrative record. Indeed Augustus' empire was far more populous than his census counts suggest, home to perhaps 50 million people. Around the same time, the first truly reliable census of Han China recorded 59,594,978 people, a number remarkably consistent with modern estimates.

Ancient censuses differed from modern ones in another way: They were usually used to establish and maintain individual entitlements or obligations, of taxation or conscription. People did not like being counted: It was often in their interest not to be, and they might avoid it if they could. But when modern states established national censuses in the 18th and 19th centuries, those individual obligations were left behind. Under the influence of "political arithmeticians” and later statisticians, the census became a scientific, statistical instrument. In the United States, it became an instrument fundamental to the operation of representative government.

Over the 19th century, ever more questions were added to the US census. Coupled with a growing population, old methods of counting could no longer keep up. Census taking had become a formidable engineering problem. In 1890, the census saw perhaps its greatest and most influential technological innovation: counting by electric machine. The “Hollerith tabulator,” named for its inventor, a New Yorker named Herman Hollerith, was about the size of a writing desk. A tall cabinet stacked at its rear gave it the overall shape of an upright piano. The cabinet displayed 10 dials arrayed in four rows and 10 columns. Each dial had 100 subdivisions and two hands, like a clock, which together could count up to 10,000.

The machine was operated by a seated clerk. At the clerk’s right, on the desk’s surface, lay a sturdy contraption with a smooth wooden handle, which Hollerith called the press. At the left was a stack of stiff cards, each one representing a person, with round holes punched out to represent different characteristics of that person: black or white, male or female, single or married, literate or not.

The operator placed each card, in turn, on the lower surface of the press and then pulled firmly down on the handle. As the jaws of the press came together, spring-loaded pins pushed down against the card. Some were blocked, while others passed through holes, making contact with cups of mercury below, closing electric circuits and advancing dials corresponding to the holes.

For Frederick H. Wines, a census employee who saw the machine in operation, this process of counting and sorting people by electricity approached a religious experience. “Under the mysterious influence of the electric current running through the machine, they organize themselves, as though possessed of volition … I can compare this current to nothing less intelligent and powerful than the voice of the archangel, which, it is said, will call the dead to life and summon every human soul to face his final doom.”

Even before the 1890 count was complete, Hollerith was selling his machines to census bureaus in other countries and, increasingly, to large businesses. In 1911, age 51, he sold the firm he had founded. It merged with three others that made complementary equipment, becoming, for a time, the Computing-Tabulating-Recording Company. That name, always awkward, was soon inaccurate too. Punch card tabulation was useful in applications far beyond the census and quickly became the conglomerate’s main business line. In 1924, the company adopted a sleek new name, befitting a new global century. Henceforth it would be known as International Business Machines—IBM.

For the next 60 years, the census remained at or very close to the forefront of information processing. Few data sets could compare with the census of a large country like the United States. During World War II, military applications—code-breaking, the computation of artillery tables, calculations for the Manhattan project—finally surpassed the census at the forefront of information processing. Even then, when Univac I, one of the first commercially available electronic digital computers, went on sale in the United States in 1951, the Census Bureau was its first customer.

The punch card changed not just the census but the entire relationship between the individual and central government. For thousands of years it was possible for a person to remain effectively invisible to distant seats of power. If, in 1880, a politician or bureaucrat in Washington, DC, wanted a list of every noncitizen living in the United States, it simply wasn’t possible, at least not without dedicating a substantial workforce to sort, by hand, through millions of written census returns. After 1890 it became a theoretical possibility, and by the middle of the 20th century a real one. Where once an insurmountable technical barrier stood in the way of centralized mass surveillance, after Hollerith only legal and ethical restraint remained to prevent it.

The punch cards are gone, but the census today is not so different from its 19th- and 20th-century predecessors. It is still a process of collecting, summarizing and analyzing information about every individual in this country. For sure, there were challenges associated with building, for the first time, a census website that could reliably handle millions of transactions each day. (Australia’s Bureau of Statistics failed embarrassingly to meet such a challenge in 2016). But in the age of Google and Facebook, these are now far from the cutting edge of information technology. Moreover, a significant proportion of households, perhaps 40 percent, are still enumerated in person after failing to respond online, by mail, or by phone.

Ironically, the information society may be what ultimately kills the traditional census. Counting people the old-fashioned way, in the 21st century, has the appearance of an increasingly expensive luxury. The census is vulnerable to the complaint that much of the information it collects—the number of people, certainly, but also their characteristics like sex, age, marital status, and educational attainment—is already recorded in other files and databases (albeit often incompletely and inaccurately). Politicians naturally ask why this special, costly exercise is necessary when other government agencies—not to mention Facebook—already hold this data.

Globally, of course, the traditional census is far from obsolete. Much of the world is not yet awash in information as the United States is. In-person enumeration remains the only viable way to reach most people in places like Bangladesh or Sudan. In the roughly 70 countries with adult literacy rates below 90 percent, even self-enumeration would be difficult. Many countries have poor communications infrastructure or limited government capacity, and while a traditional enumeration is logistically complex, it has a well-understood, robust methodology that can work nearly anywhere. That doesn’t mean no innovation is possible: New technologies like satellite imagery for address mapping and handheld tablets for recording data have the potential to lower the cost of even a traditional field survey.

Some countries are betting on a different approach to reduce the burden of census taking: greater use of so-called administrative data. This term refers to data already collected in the course of government processes—for example, postal records, tax returns, immigration files, and pension or social security accounts. The proponents of such an approach argue that it is easier and cheaper—and much less likely to arouse complaint—to simply reuse data in existing systems than it is to present each person with a blank form every 10 years.

Census takers have been studying this kind of data for decades, using it, for example, to estimate undercount error in the census. But there are hurdles to using it as a wholesale replacement for survey data. In the past it has been difficult to reliably link records for the same person in disparate sources—for example, tax returns and school records. Many countries, too, have had legal restrictions on this kind of matching, especially when the underlying records are managed by different agencies. But modern computers allow more powerful matching, while better statistical techniques have been developed to minimize the impact of mismatches. Exemptions are increasingly being crafted to allow matching for statistical purposes.

In the United States, the Census Bureau is hoping to use administrative data to reduce the need for in-person nonresponse follow-up on the 2020 census. If COVID-19 continues to delay and disrupt the census field work, the Bureau may have to make much more extensive use of such records than was originally planned. Whether existing administrative records could entirely replace a survey-based census remains something of an open question. In the US, existing government databases simply aren’t designed for this purpose.

In some countries, however, they are. In the 2010 round, 19 European countries used a “population register” to replace some aspects of a traditional enumeration. Several countries outside Europe, including India and Turkey, have also begun to establish population registers. What distinguishes a population register from other administrative databases and makes it a suitable basis for population statistics is that it includes everyone resident in a country, it is kept continuously up to date, and it can be linked to other government databases and surveys.

But to remain accurate for this purpose, a population register requires mandatory reporting when, for example, a person changes their address. In many countries, including the United States, there is a deep reserve of resistance to such reporting mechanisms—a resistance that the traditional census, with its more limited aims, is often spared. Opponents see a slippery slope leading from mandatory registration and national identity cards to laws requiring such cards to be carried at all times and police checkpoints on street corners.

It’s possible that this cultural aversion to registration is weakening. A poll taken in the wake of the September 11 attacks found a slight majority of Americans in favor of “a law requiring all adults in this country to carry a government-issued national identification card.” No such card was introduced, but federal involvement in driver licensing has partially nationalized what was previously a state responsibility. In a world where terrorism, illegal immigration and, now, suddenly, public health remain issues of serious popular concern, it’s not hard to imagine public opinion shifting further in favor of numbers, cards, and registers.

The traditional, decennial census is almost certainly in the early stages of decline. It’s surprising, really, that this curious invention, with its ancient roots, has somehow survived into the 21st century. In most countries, it will probably, eventually, be replaced by more extensive administrative records. Change may be gradual. Statisticians are by nature fairly conservative, and whatever financial or political pressures they face, today’s census takers are keenly aware that they are the custodians of a centuries-old tradition. In some countries, the United States most prominently, the glacial pace of legislative or constitutional reform will ensure the continuation of some sort of traditional census for some time.

But even with gradual change, it is possible that the number of countries taking a traditional census will peak in 2020 and fall in future rounds. It’s hard not to be sentimental about that. There is something noble about this exercise in which we line up for enumeration, not because we will individually benefit or, really, suffer if we don’t, but because we—most of us, still—believe in government on the basis of fact rather than prejudice or guesswork. There is something admirable, if a little quixotic, in the attempt to reach every person, knock on their (increasingly virtual) doors, and say to each, “You, too, count.”

When you buy something using the retail links in our stories, we may earn a small affiliate commission. Read more about how this works.