HTML( <link rel="stylesheet" href="tutorial.css" type="text/css" >)

NewMMS(<|RCode|>~~<|\begin{lstlisting}[language=R]
$1\end{lstlisting}

|>~~
              <|<div class="rcode"><pre><code>$1</code></pre></div>
|>)


NewMMS(<|SpecCode|>~~ <|\begin{lstlisting}[language=]
$1\end{lstlisting}

|>~~
                 <|<div class="spec"><pre><code>$1</code></pre></div>
|>)


NewMMS(<|InR|>~~ <|\vskip \baselineskip \hrule\vskip.1mm\hrule\lstinputlisting{$1<||>.R}\hrule\vskip.1mm\hrule \vskip \baselineskip
        
|>~~
            <|<div class="rcode"><pre><code># $1<||>.R

m4_include($1<||>.R)</code></pre></div>
|>)


NewMMS(<|InSpec|>~~ <| \vskip \baselineskip \hrule\lstinputlisting{$1<||>.spec}\hrule\vskip \baselineskip

|>~~
               <|<div class="spec"><pre><code># $1<||>.spec

m4_include($1<||>.spec)</code></pre></div>
|>)


Raw data is often rife with missing items, logical errors, and sensitive information.
To ignore these issues risks alienating respondents and data users alike, so data
modification is a necessary part of the production of quality survey and census data.
Tea is a statistical library designed to eradicate these errors via a framework
that is user-friendly, effectively handles Census-sized data sets, and processes
quickly even with relatively sophisticated methods.

Paragraph(Some sample use cases)
Items(
∙ A record is missing age, so it gets sent to the imputation module to draw values of
age given schooling (if available for the record) and neighborhood characteristics. Did
the imputation generate a married 12-year old? Each draw is sent to the editing module to 
verify every consistency check provided by the user, and strike out failures.

∙ One record might be for a 110-year old. This is distinctive enough that publishing a table
of incomes by age might publicly reveal this person's income. Tea provides a module for
finding distinctive cases and blanking out the most distinctive characteristic. Of course,
the record then needs to be sent to imputation, and the imputed value(s) need to be
edited.

∙ Crosstabs giving age $\times$ race $\times$ income and are cleared for
disclosure concerns, but users want to use statistical techniques based on record-level
data, not crosstabs. Tea's raking module generates a synthetic data set that is as close
as possible to the constraint of the crosstab (and, of course, meets all constraints).
)


HTML(You can also read Link(http://ben.klemens.org/pdfs/tea_tutorial.pdf~~this tutorial as a PDF).)
TeX(You can also read this tutorial in your browser at \url{http://b-k.github.io/tea-tutorial/}.)

Items(
∙ TeX(Section) Ref(Basics~~basicssec) presents a `hello, world' example to introduce the basic mechanics of the system,
and the format of the spec file used to describe a survey.

Comment(
    ∙  Because Tea uses R as a front-end, we include TeX(Section) Ref(a brief
    overview of using R~~rsec) for the purposes of querying, subsetting, and summarizing
    tables. )

    ∙ TeX(Section) Ref(Recodes~~recodesec) describes Tea's syntax for generating new variables based on other variables, 
like tt(is_heterosexual_marriage) calculated from the sex of the spouses, or log of income calculated from income.


    ∙ TeX(Section) Ref(Imputation~~impsec)
    goes into greater detail on the procedure for imputing missing values, the specific
    models available, and the use of multiple imputation.

    ∙ TeX(Section) Ref(Editing~~editsec) describes em(editing), the process
    of finding elements that fail validity checks.

    ∙ TeX(Section) Ref(Disclosure avoidance~~dasec) describes a table-based method of finding respondents at
    risk of being identifiable, and blanking out portions of their information so an imputed
    value can be filled in.

    ∙ TeX(Section) Ref(Synthetic microdata~~synthsec) describes Tea's mechanism
    for generating synthetic microdata with marginal values identical to the source
    data. This is useful for running bootstrap-style tests and protecting the identities
    of individual respondents.

    ∙ HTML(An) TeX(The Appendix in Section) Ref(appendix~~appendices) provides some
    other useful information, including some notes on missingness markers in various
    systems and a suggestion on how to cite Tea in your papers.
)

bf(This tutorial is a work in progress.) Tea has an open development model, wherein users
have been able to download versions almost since its inception. We follow the same process
for this tutorial, posting the draft as we write it, and adding more every week.


Paragraph(Installing Tea) The interface for Tea is an R package, but depends on several
C libraries.
That means that installation is not as trivial as typical R packages, but we found
the large-scale processing we had to do to be largely impossible when staying exclusively
within ecosystem of things installable via tt(install.package).

If you are set up to use virtual machines, check out the
Link(https://github.com/GovReady/govready-Tea~~GovReady Tea machine).

Installation is easiest on a system with a package management system.
Package managers give you immediate access to thousands of programs, libraries,
language compilers, games, and everything else a computer can do. Linux users
know their package manager; Mac users can use Link(http://macports.org~~MacPorts)
or Link(http://finkproject.org~~Fink); Windows users can use
Link(http://cygwin.com~~Cygwin).

Once you have the basic platform set up,

Items(
∙ Install the Link(http://apophenia.info/setup.html~~Apophenia library)
∙ Install the R-to-Apophenia link, Link(https://github.com/b-k/Rapophenia~~Rapophenia)
∙ Install Link(https://github.com/rodri363/tea~~Tea)
)



Chapter(System basics~~basicssec)

The full specification of the various steps of Tea's process, from input of data to
final production of output tables, is specified in a single plain text file, herein
called the em(spec file).  An R script executes the plan based on the spec file description.

There are several benefits to such a work flow.  First, because the spec file is separate
from programming languages like R or SAS, it is a simpler grammar that is easier to
write, and analysts whose areas of expertise are not in programming can
write technical specifications without the assistance of a programmer or training
in a new programming language. Second, this follows the programming adage that data and
procedure should be kept separate. Third, we hope that spec files will be published
along with data sets, so that users can see the assumptions made during processing.

Throughout this tutorial, code snippets that have a file name can be found in the 
Link(https://github.com/b-k/tea-tutorial.git~~tutorial repository).
In the sidebar on that page, you will find a link to download a zip file of the full set
of files, or users familiar with git can clone the repository.

Being plain text, you can open and edit the files using any text editor (Notepad,
Kate, vi, ...). From that directory, you can cut/paste the R sample code onto the R
command prompt, or run the program using R's the tt(source) command, e.g., from R:

RCode(
source("hello.R")
)

This leaves you at the R prompt with the Tea library loaded and any variables defined in
tt(hello.R) ready for your interrogation.

If you want to run a script and exit, you can do this via R's tt(-f) flag. From the
command line:

SpecCode(
R -f hello.R
)


Paragraph(Hello)

Our first example will load in a data set (the Public Use Micro Sample for the American
Community Survey, DC, 2008) and do some simple queries, without yet
doing any editing or imputation. Here is the spec file:

InSpec(hello)

Almost everything in the spec file will consist of key:value pairs, which may be
organized into groups (and subgroups).

Everything on a line after a tt(#) is a comment that the spec parser will ignore.
Blank lines are also ignored.

The first non-comment, non-blank line of every spec file needs to specify a database;
more on this requirement in TeX(Section) Ref(Subkeys~~subkeys). This will be a file generated on
disk (or opened, if it already exists), using the SQLite database library. The name
doesn't have to end in tt(.db), but we recommend sticking to that custom. If you are
an SQLite expert, you can view or modify this database using your favorite SQLite tools.

If your data is already in an SQLite database, then you can skip the tt(input) step, but
in typical cases your data will be in a plain text file. These are typically called CSV
files (for comma-separated values), though the delimiter can be set to almost anything;
we prefer pipes, | .

The tt(input) segment of the spec gives the name of the input
file, and the name of the table that will be generated from that input.

This spec file tells Tea to overwrite the tt(dc) table if it already exists. This can be
useful for running an analysis from start to finish, but in most cases, the read-in is
time-consuming and most later operations will not modify the source data, so the default
if the input segment has no tt(overwrite) key is to assume tt(overwrite: no). More on this
below.

The spec also includes a section describing two of the fields: tt(age) is a real number, and
tt(schl) is an integer from 1 to 24. Not all fields have to be specified, but it helps to
guarantee that SQLite doesn't treat the entries as text, and will be required when we get
to editing. You can have one tt(fields) section for every table, or several which will be
treated as a single unit.

Spacing in the spec file is largely irrelevant, but each key:value pair must end with
a newline (or the \} closing a group, though we think having the \} on its own line
looks better). You will see some examples below where the expression is too long for
one line, so lines are joined by ending the first line with a backslash.

Now that we have specified the characteristics of the survey, we can move on to
tt(hello.R), which makes use of that description:

InR(hello)

Items(
    ∙ The first line, tt(library(tea)) loads Tea. All functions defined therein can now be
        used at the R command line.

    ∙ The second line reads the spec file, tt("hello.spec"). 

    ∙ At this point, the database is open, and Tea knows about all of
    the configuration options in the spec file. Thus, the tt(doInput()) function
    does not need input arguments specifying the name of the text file or other details—that
    can all be read off from the spec.

    ∙ After tt(doInput()) runs, the requested table, tt(dc), exists in the database. The
    tt(teaTable()) function pulls data from the database into an R data frame. In the
    example here, the first use of tt(teaTable) will print to the screen, so limit
    the number of columns to pull to 20, and request only three columns.

    ∙ The second example uses R's tt(summary()) function to show the means and medians of the
    age and schooling categories.
)

We do not print the unremarkable output from running tt(hello.R), but you are encouraged
to make sure that your Tea installation is working by running it yourself.

Section(Subkeys~~subkeys)
Why does the first line of the spec file have to be a database? Because the first step in
processing is to convert the spec file into a database table. The table is named tt(keys),
and you can view it like any other:

RCode(
teaTable("keys")

TeX(<|)\begin{verbatim}|>)
                 key   tag count          value
1           database 0 tag     1        test.db
2   input/input file 1 tag     2 dc_pums_08.csv
3 input/output table 1 tag     3             dc
TeX(\end{verbatim})
HTML(<|)|>)


The format of the tt(keys) table is subject to change, so please do not rely on it for
processing. However, it can be useful to check that the parser did what you expected.
The tt(fields) and tt(checks) sections do not get encoded into this table, but every other
key:value pair will get one row. Comparing this table to tt(hello.spec) shows how the
keys embedded in a group are written via a tt(group/key) notation. For example,
tt(input/input file) appeared in the spec as 

SpecCode(
input {
    input file: ...
}
)

That said, here is an Link(http://rodri363.github.io/tea/keys.html~~alphabetical list of all Tea keys).



Chapter(Recodes~~recodesec)
A em(recode) is a variable that is deterministically generated from another variable. The
name comes from variables made via simple code switches, like taking three-digit race codes (100, 101,
..., 200, 201, ...) and reducing them to one-digit race codes (1, 2, ...). But
they are a versatile solution to many little survey annoyances. If you find yourself
referring to a complex expression several times, it might be best to define a variable
based on that expression. It is easy to use a recode to define sets of observations
that can then be treated as separate categories for imputation modeling.

There may be characteristics of the group, such as the householder's income and the total
income for all members of the household, that are useful for processing each record.
Tea handles such characteristics by adding a field giving the group-based value to each
record.

On the back-end, the recodes are generated via an SQL table view. We have a table named
tt(dc), so the recodes will be in a table named tt(viewdc). Therefore, all recodes are
simple fragments of SQL.

Here is a sample spec file that generates several recodes.

InSpec(recode)

Items(
    ∙ For imputation, we will need a column with a unique identifier, so Tea can record
    which imputed value goes with which record. Census household data usually gives a
    household number (here, tt(serialno)) and a household member number (here, tt(sporder)),
    so generate a unique ID by joining the two together.

    ∙ Income in US dollars is given by the tt(PINCP) variable. Because it is often easier to
    work with log of income, generate a variable with log of income. The tt(+10) prevents
    log-of-zero errors.

    ∙ The next recode segment works in groups. Each value of the given tt(group id) is
        treated as a unit, and the given value is written down for every member of the group. 
        So after the recode step, every observation will have a tt(hh_in) field giving the
householder's income and a tt(house_in) field giving the total income for the household.

    ∙ Calculate the householder income by taking the maximum over PINC for the householder (i.e., the person for whom tt(sporder==1)) and zero for 
        everybody else. We use this form
        often to pull the characteristics of one household member.

    ∙ If you think the Link(http://stackoverflow.com/questions/4622/sql-case-statement-syntax~~SQL syntax
    for case statements) is awkward, we agree with you. Watch out whether there is a
    variable name after the tt(case) keyword [Tt(case pinc when ...)] or not [Tt(case when ...)].

    ∙ A recode segment can not refer to fields generated in that recode segment. However,
        subsequent recode segments can refer to variables generated in previous segments. 

    ∙ Because SQL tt(case) statements are difficult to write, but recodes with an
    if-then format are so common, Tea provides a special syntax.
    The tt(has_income) variable will be zero if tt(PINCP) is zero, and will be one in all
    other cases. The tt(age_cat) recode behaves similarly: if tt(AGEP <=15), then
    tt(age_cat) is set to zero, and so on. The tt(between) keyword is standard SQL,
    and is inclusive of the endpoints of the range. The default of tt(X) will be
    assigned for any negative or NaN values.

    ∙ The spec could also have several tt(group recode) segments. For example,
        one segment could give state averages and one could give county averages.
)

The R script uses some of those recodes.

InR(recode)

Items(
    ∙ The tt(doInput) function does the recodes, so there is no need for a separate tt(doRecodes) function.

    ∙ We can treat the new fields exactly like we do the ones supplied in the input data.

    ∙ Tea keeps a few variables in an R environment named tt(teaenv). The name of
        the table after the last step is stored in tt(teaenv$active_tab). If there
        were no recodes, this would be tt(dc); because there were recodes, this is
        tt(viewdc). More on tracking the flow of tables below.

    ∙ The script gets the average log income and average tt(log_house_in), taking care to
        add a condition to only get those records where that sum is not tt(-Inf).
        But the average of tt(log_house_in) is an unusual statistic, because it goes per
        capita: a household of seven counts seven times as much as a household of one.

    ∙ The last line of the script gets an average using one value per household. It uses
        the tt(dbGetQuery) function from the tt(RSQLite) library to directly query
        the database. The function needs a database connection to talk to, and that too is
        in the tt(teaenv) environment, stored as tt(teanev$con).
)


Paragraph(The flow of inputs and outputs)

With a few exceptions, every segment can have tt(input table), tt(overwrite), and
tt(output table) keys. Keeping track of these things will become important as more
segments are added and if multiple tables are being used.

If these keys are not given, the output table from the previous segment of the spec will
be used as the input table for the next segment.

If you do not specify the output from the recodes, then the output table name at the
end of a sequence of recodes is the input table name with tt(view) at the front. E.g.,
tt(intab) $\to$ tt(viewintab).

The imputation uses tt(filled) as the default output table name, but you are encouraged to
use something else.

The flow is thus typically a chain of steps, like input $\to$ recode $\to$ join with
another table $\to$ impute. If you give an instruction to overwrite input every time,
then the recode, join, and impute steps will also rebuild their output tables. If
you remove the input table (SQL: tt("drop table read_in_data")) then the table will
be re-read and all subsequent steps rerun.  The exceptions: tt(checks) and tt(fields)
apply to all tables, the tt(input) segment takes an tt(input file), and the imputations
are always re-run.

As above, the R variable tt(teaenv$active_tab) keeps track of the latest output table.


Chapter(Imputation~~impsec) 

This section will begin with an overview of the many methods one could use to fill in a
blank in a survey, and will then show how Tea implements them.

The first are what we will call mechanical edits, which are deterministic if-then
rules.  For example, if a birth date is given but an age is not, then fill in the age
by calculating the time span between the birth date and the survey.

Paragraph(Mechanical edits)
Mechanical methods are best when there is a single correct answer that can be calculated
from available information, such as in an accounting ledger where certain sets of numbers
must add up to certain sums.

Here is a simple case from the example below:

SpecCode(has_income=0 and PINC is null => PINC=0)

This reads as an if-then statement: if tt(has_income=0 and PINC is null), then set
tt(PINC=0).

People at extreme ends of a range, such as the exceptionally long-lived or the
exceptionally high-income, are a disclosure avoidance risk, so statistical agencies often
em(top code), changing all values above a threshold to the threshold itself. To set
the income of all people making over a million a year to exactly a million:

SpecCode(PINC>1e6 => PINC=1e6)


Some history and literature: Citet(edit:review~~Pierzchala) gives an overview of the problem of
editing (including a useful glossary). The models covered in that paper are uniformly
in the class of mechanical edits, including several implementations of the method
of Citet(fellegi:holt~~Fellegi and Holt) and deterministic nearest-neighbor methods, such as that used
by Citet(garcia:balance~~García).

This thread has continued into the present day.
Citet(chen:threesystems~~Chen) discusses three newer systems, each of which focuses on
determining the cause of the error and imputing accordingly: the editing system for the
American Community Survey (ACS, from the US Census Bureau) consists of a long sequence
of if-then rules that specify what substitutions are to be made given every type of
failure; the DISCRETE system Citep(winkler:discrete) uses Fellegi-Holt's method to
derive imputation rules from the edits; what is now CANCEIS Citep(bankir:canceis) uses
a relatively sophisticated nearest-neighbor rule for imputation.

Paragraph(Probabilistic edits)
Probabilistic edits assert some underlying distribution
or model to the variables, and fill in a missing value by making a draw (sometimes
multiple) from the underlying distribution.

The incumbent method is em(Hot Deck), a nonparametric method in which nearby nonmissing
values are used to generate imputed data Citep(cresce:census). In its simplest form,
Hot Deck imputation for a missing value consists of substituting a neighbor's value
for the missing value. This is a mechanical imputation: if the value is missing, then fill
in using the neighbor's value.

Randomized Hot Deck is probabilistic: select a subset of similar respondents (see below),
and assume that the missing value takes on the same distribution as the values in the
respondent set. Having established this empirical distribution of values, make a draw (or
draws) to fill in the missing value.

Hot Deck does not make assumptions about mathematical relations among variables. For
example, there is no need to evaluate and substantiate claims that variables are
multivariate Normal, or there is a linear relationship between some set of dependent
variables and independent variables. But Hot Deck does make the basic modeling assumption
that neighbors are similar, and that values observed in the neighborhood are the most
likely values that would have been observed had the missing data been gathered.

Other models make other assumptions. We may assume that income has a Lognormal
distribution, in which case we can calibrate the distribution by calculating $\mu$
and $\sigma$ using the known values from the respondent set, and then drawing from
the estimated Lognormal distribution.

The em(EM algorithm) described below builds and draws from a multivariate empirical
distribution, akin to the univariate Hot Deck distribution.




Paragraph(Categories)
There is a broad intuition that imputed values should be taken from records `close' to the
one missing information, perhaps in the physical sense, or in a broader sense, like
how we expect an apartment dweller to be more like another apartment dweller than a similar person
living in a single-family house, em(ceteris paribus).

Tea thus provides a mechanism to do imputations by categories. Given this part in the
tt(impute) segment of a spec file,

SpecCode(
    categories {
        income_cat
        sex
        puma   #public use microdata area
    }
)

and a record with a known tt(income_cat), tt(sex), and tt(puma), only records with the
same characteristics will be used to train or fit the model.

These categories `roll up' if there is insufficient data. If there are not enough
observations in the tt(income_cat) $\times$ tt(sex) $\times$ tt(puma) category to build a model,
then the last categorization in the list is thrown out, and the model fit using
tt(income_cat) $\times$ tt(sex). The roll-up could continue until there are no categories
left, in which case the entire data set is used. Set the minimum group size before
a roll-up occurs via an tt(impute/min group size) key.

Comment(
The EM algorithm handles continuous variables internally (see below), but most
methods are strictly categorical. But one could easily generate a recode, like the
tt(income_cat) recode above, to group the continuous variable into ranges.
)


Paragraph(Imputing incomes~~incimp)
The following example imputes missing incomes using two methods.

Here is the spec file, with several imputation methods:

InSpec(incomes)

The spec uses two features to reduce redundancy.
Items(
∙ At the top of the file, the tt(include) line tells the parser to paste the full contents
of the listed file  into the current file at this point. We can thus
reuse all of the declarations and consistency checks listed in
tt(recode.spec) above.

∙ The tt(paste in) lines do a similar thing, but for groups. In this case, Tt(paste in:
   common) instructs the parser to find the group defined to that point with the name
tt(common) and insert all key/value pairs defined in that group at this point. This makes
it easy to write two imputations that have most of their specification in common.
)

Please note:
this is a tutorial consisting of simple demo code, so we make no promises that this
is a good model for imputing missing data. We do not know of any Census surveys that
use these models for income imputation. But we do believe that the spec format
makes the assumptions underlying the imputation clear, and provides an easy means of
quickly trying different models.

After every value is imputed, the edits are checked. Above, you saw that tt(recodes.spec)
has an edit specifying that if tt(has_income=0 and PINC is null) then set tt(PINC) to zero. This
guarantees us that at the end of the imputation of tt(has_income), the only people
with incomes marked as missing are those who are imputed as having a nonzero income.



Paragraph(Check out any time you like)
Tea is intended for multiple imputation, either in the formal sense (see, e.g.,
Citet(schafer:multi~~Schafer)) or in the sense of this example, where multiple models
are used on the same data set.

To facilitate this, imputations are not written to the tt(input table) specified in the
tt(impute) section, but to a separate tt(output table).  For the tt(has_income) variable,
the output table was tt(has_in), so after the imputation, we can have a look at that
table:

RCode(
teaTable("has_in", limit=5)

TeX(<|)\begin{verbatim}|>)
  draw value       id      field
1    0     1   348902 has_income
2    0     0  6462602      PINCP
3    0     0  6462602 has_income
4    0     0 18435903      PINCP
5    0     0 18435903 has_income
TeX(\end{verbatim})
HTML(<|)|>)

Each id/field coordinate in the original table that has an imputed value will have a row
(or rows)
in the fill-in table. Here only one draw was made per imputed value, so the tt(draw)
column is all zeros; if you specify tt(draw count) to be greater than one, then this will
take appropriate values.

This is the output for the imputation of tt(has_income),
but because the edit rules specify that tt((has_income=0 and PINCP is NULL)) is an inconsistent
value, the fill-in table also specifies cases where PINCP has to be adjusted to maintain
consistency.

The list of imputations is sometimes useful for debugging, but we actually want the complete
data set. Check out an imputation into a new database table as needed:

RCode(checkOutImpute(teaenv$active_tab, "complete_tab", filltab="has_in")
outdata <- teaTable("complete_tab")
)

One imputation often follows another. Here, the model of income depends on the
tt(has_income) variable, so we have to impute tt(has_income) first. Imputations are
done in the sequence listed in the spec file. Use the tt(earlier output table) key in
an tt(impute) group to specify a table of fill-ins to be checked out before embarking
on the imputation described by this group.

If you want to not bother with a fill-in table and write imputations directly to the
input table, add the tt(autofill) key to an imputation.

Paragraph(R code for imputing income)

Here is some R code to 

Items(
∙ Blank out some incomes, adjusting and saving the data before doing so.
∙ Do the several imputations as per the spec file above, via tt(doMImpute).
∙ Check out the imputations in sequence, bind them together, and plot the several log income
distributions on one plot.
)

The sample data used in this tutorial is already-cleaned public use data, so the only
missing incomes are for people under 15 (who all have missing income, which tells us
that the ACS probably has an edit akin to tt(age<=15 => PINCP=NULL)).
The tt(pokeHoles) function
first records the original distribution as reported by the Census (in the tt(precut)
table), then
blanks out the incomes of about 30\% of respondents (probably including a good number
of already-blank under-15s). Because tt(pokeHoles.R) is not really
about Tea, we do not print it here, but you can inspect it on the
Link(https://github.com/b-k/tea-tutorial/blob/master/pokeHoles.R~~pokeHoles.R page) of the repository.

This tutorial will not go into detail regarding the especially R-heavy portions of
the code, such as how the tt(ggplot2) package was used to generate the final plot, though
the comments give a sketch of what the code is doing.

InR(incomes)

TeX(Figure) Ref(The figure~~log_wageplotspic) presents the final graphic. The data
used for imputation is not pictured, but hot deck makes draws from that distribution,
so the hot deck curve could be read as the post-cut data set. The imputation using a
lognormal distribution produces a distribution that is lower at the peak and smooths
out some of the inflection points in the data.

If we believe that people at extremes of the income spectrum are underrepresented in the
complete data, then the fact that Hot Deck simply replicates the nonmissing data may not
be desirable.


Pic(log_wageplots.png~~Log wages from the data and imputed using Hot Deck and a Normal distribution.~~90~~.5)

Chapter(Imputation with auxiliary data~~emsec)

Consider two data sets that ostensibly measure the same thing, such as American Community
Survey income questions and income questions on tax returns. The biases and
assumptions underlying both questions are slightly different. People have a strong
incentive to understate taxable income, or might treat exceptional circumstances
differently between the two. But we have a strong expectation that the two are closely
correlated.

We can expect that for some respondents, we have both observations,
for some only one or the other, and for some we have neither.

Tea has an implementation of the expectation-maximization algorithm by Citet(Hartley:em~~hartley), popularized in Citet(rubin:implementation~~Rubin).
The end result in our two-income example is a table with ACS incomes on the rows and
IRS incomes on the columns. We expect that most observations will be along the main
diagonal where ACS income equals IRS income.

Given a person with a known IRS income, we can impute their ACS income by taking the
appropriate column and making a random draw from the density in that column. For somebody
with no income data, we can make a draw from the entire table.

This method has several advantages over substituting in IRS observations. It accommodates
the uncertainty of using an auxiliary data source, and several draws from the IRS data
would produce a spread of income values (see the discussion of multiple imputation
below). This algorithm becomes especially useful when we go past two dimensions into
six or seven, each of which is partially specified.

The primary benefit of Hartley's EM algorithm is that it uses all available information,
in a Bayesianly-correct manner, to generate the final table. If a respondent has only ACS
or only IRS data, their response is still used to generate the densities in the final table.
Tea's implementation offers a few minor benefits: it accommodates near-misses (and so is
usable for near-continuous variables like incomes) and is implemented efficiently enough
to work for up to about seven-dimensional tables (though it starts to freeze up past that point,
especially when many-valued variables like age or income are involved).

Paragraph(An example)

In the code repository, there is a second data set, named tt(fake_in.csv). This is indeed fake
data, generated via tt(fake_in.c) (also in the repository), which reads the first 2,000 income observations from the DC PUMS, and multiplies each by a draw from a truncated Normal(0.9, 0.2) distribution. Thus, the fake income data systematically under-reports the actual data.

Our first task in processing is to merge the data sets into one table. Joining data sets
is one of the great strengths of SQL databases, and we only have to specify how it is done
using a tt(join) segment in the spec file. 

InSpec(joined)

Here is the R code. The tt(library) and tt(readSpec) are by now familiar to you, but the
script then introduces a new command, tt(doTable). Because tea derived the pipeline
of input/output tables from your spec file, it knows what steps to take to generate
any given table. Inspecting the spec ourselves, we see that tt(dc_united) depends on
tt(viewdc), which is the product of recodes on tt(dc), which is a read-in from the
tt(dc_pums_08.csv) file, and on tt(fake_incomes), which is a read-in from tt(fake_in.csv).
The tt(doTable("dc_united")) function will therefore implement all of those steps.

As above, the public-use microdata is complete, so we need to use the tt(pokeHoles)
function to produce some missing data. The script then calls tt(doMImpute).

InR(joined)

Here is the output. Note that, even though the fake data is biased, generated via a
process that understates income by 90% of the true values, using it as a covariate gave us
better results than not using it.


RCode(
[1] "Log diffs for income imputed via EM algorithm (w/aux data)"
abs(dcu.pincp - imp.pincp)
Min.   : 2.303
1st Qu.: 7.251
Median : 8.701
Mean   : 8.197
3rd Qu.: 9.799
Max.   :13.715

[1] "Log diffs for income imputed via Hot deck"
abs(dcu.pincp - imp.pincp)
Min.   : 2.303
1st Qu.: 9.623
Median :10.483
Mean   :10.166
3rd Qu.:11.296
Max.   :13.635
)


Chapter(Editing~~editsec) 

To this point, we have covered imputation of missing values, where those values and
edit-associated values get checked. This segment will cover editing the data that is already
part of the data set. There are three things that can be done when an edit is hit:

Items(
    ∙ Keep a tally of which fields in which records failed how many edits.
        If you only want to get a count of how many fields fail in how many records,
        try the tt(CheckDF) function.
    ∙ Mechanical edit: make a deterministic change you specify given that the edit was triggered.
    ∙ Blank it: use a heuristic specified below to blank out a field, so you can impute it
        in a subsequent call to tt(doMImpute()).
)

The sample spec for this part of the tutorial will focus on age, income, and schooling,
and the presumption that children do not have advanced degrees or significant incomes.

InSpec(edit)

The tt(checks) section looks much like the sections above. One edit, that if age is
less than thirteen and income is positive, has no associated mechanical edit, because we do
not know whether a fault is in the age or the income field. The other edit has an
associated mechanical edit, that if years of schooling look off, we should assign the
U.S.-typical age minus five to schooling. As with most deterministic edits, this
comes off as a somewhat strong assumption.

Any time a record needs to be verified, including during edits, these checks will run.

Comment(
Note that edits are pessimstic: put the event that would cause a failure in the tt(checks)
section of the spec (e.g., tt(PINCP<0)), not the condition that you would like to see
remain true (e.g., tt(PINCP>=0)).

You already saw that the imputations always check their results against the edits. For
example, a Normal distribution could easily generate negative incomes, but Tea checks
every draw against every consistency check, and throws away and redraws any that fail.
)

This spec file also has an tt(edit) segment, which allows you to describe similar
logistics to the tt(impute) segment, like the input table (which is assumed to be the output
from the last segment of the spec), the output table, and whether to overwrite.

That segment gets used when you call tt(doEdit()) from R. Here is a sample program. 

InR(edit)

The tt(doEdit) call will fist handle the mechanical edits.
When a record hits a failure with a mechanical edit:
Items(
    ∙ The change made by the edit is recorded in the output table.
    ∙ For the purposes of this record, this modification is exhausted. The edit (here, that
        tt(schl>12 and agep$\leq$14) is an error) wil still be checked.
    ∙ The whole check starts over, with the newly modified record.
)

The record may encounter additional mechanical edits, or it may get to the end of the edit list
with no further changes. An exhausted mechanical edit will not be reapplied to a record, so we are
guaranteed forward progress.

But there may still be edit failures. In this case, the next step that tt(doEdit()) will
take is to blank out fields, with the intent of allowing you to impute them later.

Paragraph(The blanking procedure)
At this point in the process, we have checked every edit, and have a tally of every edit
that failed for a given record, and know which fields are associated with the edit.

We would like to blank out the value that is implicated in the most edits. Say that
an edit regarding age and housing tenure failed. If age appears in a hundred edits and
fails in six, while tenure appears in three edits but fails all three, we are inclined
to believe that it is tenure (and perhaps some other variable in other age-related
edits) that needs fixing. So we use a modified percentage of edits failed to determine
which field to blank.

The modifier is a user-specified salience, which you may set in the tt(fields) segment:

SpecCode(
fields {
    age: [1.5] real
    status: [2] cat married, single, divorced
    tenure: cat own, rent
}
)

Here, tenure has the default salience of one, and age has a salience of 1.5. In the example
above, the score for age is now 1.5 * 6/100 = .09. The corresponding score for tenure
is thus 1*3/3 = 1, so tenure is still going to be blanked if there is a failing edit
regarding tenure and age.

After tenure is blanked, all later edits regarding tenure are considered resolved for this
record.

Paragraph(The follow-up imputation and the audit trail)
At the end of the tt(doEdit()) step, we have a fill-in table named tt(edits), which is
identical to the output from the imputation steps above, except that it may include some
tt(NULL)s, indicating that a certain field in a certain record is to be blanked out.

We could 
use tt(checkOutImpute) to fold the list of changes into a new table and check that the
edits have been made.

Or, we can use the tt(previous output table) key in the imputation to have the imputation
automatically pull the edits for us.


In this setup, the edits and imputations have all been written to the same table, thus
producing a complete audit trail. We can query for every entry with a given record ID to
see what happened to the record over the course of things:

RCode(
teaTable("ed_imp", where= "id=72023405")


)






Chapter(Disclosure avoidance~~dasec)

Chapter(Synthetic data~~synthsec)
em(Raking) is a method of producing a table of cells consistent with a given list of margins.
For example, we may require that any data set we produce have an age $\times$ race
$\times$ ethnicity distribution and a sex $\times$ age distribution that exactly
matches the complete data in a census.

Raking is a simple method by which the values in the survey table can be incrementally
reweighted to match the values in the census. Let em(close) indicate Kullback-Leibler
divergence; then the raked table is the closest table to the original data subject to
the constraints of the row and column totals specified by the census.

Given more dimensions, we may want certain sets of dimensions to match a reference, such
as requiring all block $\times$ sex categories to fit to the reference values.

But consider the case where there was no survey; we can begin with a `noninformative'
table in which all cells are filled with ones, and rake that to the closest table that
matches the specified margins.

Raking a noninformative table as per the above definition (or many others) to margins
deemed to not be a disclosure risk will produce microdata that also bears no disclosure
risk, and can be presented to the public.

Tea's raking algorithm is designed to work well with sparse tables, and makes heavy use of
the database for its work. If the marginal cells imply a large number of zeros (which is a
near-certainty in high-dimensional tables), those cells will remain zero in the raked
table.




Chapter(Appendix: additional info~~appendices)
Here are some additional notes about using Tea and its attendant tools.

Section(Ways of expressing missing data) Because Tea is about dealing with missing data, it
may be worth getting to know how various systems express missingness.

Items(
∙ SQL has a tt(NULL) marker.
∙ The IEEE floating-point standard has a not-a-number (NaN) marker (in fact, billions of NaN markers).
∙ R has an NA marker distinct from its NaN marker.
)

These are explicit markers of missingess that can not be mistaken for erroneous values.
It is easy to find systems that use ad hoc markers to represent missingness, including zero, -1, an
empty string, and so on, especially in older systems that predate the IEEE standard
(first published in 1985).  We recommend using the tt(input/missing marker) key to specify
this ad hoc value; Tea will then read elements intended as blanks into the database as tt(NULL).

The IEEE 754 standard specifies a tt(NaN) marker to represent math errors such as
Tt(0./0.), and this is often used to represent missing data.Footnote(The standard also
states that Tt(1./0.) produces Tt(INFINITY) and Tt(-1./0.) produces Tt(-INFINITY).)

Comparison to tt(NaN) em(always) fails---the standard specifies that even tt(a==a)
is false if tt(a) is tt(NaN). Therefore, there are functions to use to check for
missingness. in SQL, 
SpecCode(a is null
or
a is not null) are correct ways to check for
missings or nonmissings. Note that SQLite allows either tt(NULL) or tt(null). In R
RCode(is.na(a)) will check whether tt(a) is tt(NaN).


Especially careful readers will note that using tt(NaN) as a missingness marker is
potentially ambiguous: a tt(NaN) could represent truly missing data or
it could represent the result of dividing a nonmissing zero by another nonmissing
zero. Thus, R has an tt(NA) marker, which is distinct from its tt(NaN) marker.
The authors of this document have surveyed R users, including the authors of some
popular packages, and have not found a single user who takes care to preserve the
distinction between tt(NaN) and tt(NA) in practice. Note that tt(NaN)s are a subset
of tt(NA)s, not the other way around:

RCode(
> a<-NaN
> is.na(a)
[1] TRUE


> b<-NA
> is.nan(b)
[1] FALSE)

Therefore, we recommend using the shorter tt(is.na), which catches both tt(NA)s and tt(NaN)s.

See also Link(http://modelingwithdata.org/arch/00000132.htm~~this blog post) for bit-level
discussion of the types of tt(NaN).


Section(Using m4 to reduce repetition)



Section(Citing Tea) To date, the only formal publication describing Tea is a UN
proceedings paper Citep(klemens:tea), so until we get out more publications it is may
be the best option when citing Tea in an academic paper:

SpecCode(
@inproceedings{klemens:tea,
    author={Ben Klemens},
    title= {Tea for Survey Processing},
    booktitle={Proceedings of the Conference of European Statisticians},
    chapter={Working paper 29},
    crossref={UNECE12}
}

@proceedings{UNECE12,
    title={Proceedings of the Conference of European Statisticians},
    series={Work Session on Statistical Data Editing},
    organization={United Nations Economic Commission for Europe},
    year=2012,
    month=Sep
}
)





Comment(

In this snippet of the tt(demo.spec) file, we specified a database to use, an input
file to be parsed, and an output table to write our imputations to. Behind the scenes,
SQL scripts and C functions are being executed. As we will see, other scripts that
are run from the spec file perform more complicated algorithms; however, before we
go through an example of a full spec file, we discuss the environment and systems in
which Tea runs and the processes underlying the spec file.

Subsection(Environment and underlying systems)
Tea is based on three systems: C, R, and SQL.Footnote(C is the successor to B, which
was the successor to BCPL: basic combined programming language. R, a successor to S,
is named after its authors, Robert Gentleman and Ross Ihaka. SQL stands for structured
query language.) Each provides facilities that complement the others:

bf(SQL) is designed around making fast queries from databases, such as finding all
observations within a given range or category. Any time we need a subset of the data,
we will use SQL to describe and extract it. SQL is a relatively simple language, so
users unfamiliar with it can probably learn the necessary SQL in a few minutes---in
fact, a reader who claims to know no SQL will probably already be able to read and
modify the SQL-language conditions in the tt(checks) sections below.

Tea stores data using an SQL database. The system queries the database as needed to
provide input to the statistical/mathematical components (R, C function libraries,
etc.).  Currently, Tea is written to support SQLite as its database interface; however
it would be possible to implement other interfaces such as Oracle or MySQL.

Output at each step is also written to the database to be read as input in the next
step. Thus the state of the data can be recorded at each step in the process to allow
for auditing of suspect changes.

Outside the database, to control what happens and do the modeling, the Tea package
consists of roughly 6,000 lines of R and C code.

bf(R) is a relatively user-friendly system that makes it easy to interact with data
sets and write quick scripts to glue together segments of the survey-processing pipeline.
R is therefore the interactive front-end for Tea. Users will want to get familiar with
the basics of R.  As with SQL, users well-versed in R can use their additional
knowledge to perform analysis beyond the tools provided by Tea.

bf(C) is the fastest human-usable system available for manipulating matrices, making
draws from distributions, and other basic model manipulations. Most of the numerical
work will be in C. The user is not expected to know any C at all, because R procedures
are provided that do the work of running the underlying C-based procedures.

%\comment{DV: Taking this out for now until we get all the other syntax settled.

%Now that we have a better idea of the environments in which Tea is run, here is 
%an example of a full tt(spec) file, tt(demo.spec):


%\begin{figure}
%\lstset{columns=fullflexible}
%\lstinputlisting{../../demo/demo.spec}
%\caption{A sample spec file that imputes an AGEP variable using WAGP, SEX, 
%and the tt(hot deck) and tt(ols) imputation models. The contents of this spec 
%file are explained below}
%\end{figure}

%}

The configuration system (tt(spec) file) is a major part of the user interface with Tea. 
As is evident from tt(demo.spec), there are many components of the spec file that all 
perform certain functions. We begin by explaining the concept of \textit{keys}.

Section(Interactive R)\label{rsec}
The specification file described to this point does nothing by itself, but provides
information to procedures written in R that do the work. In fact, Tea is simply 
called as a library in R, and the actions described in the spec file are executed by
issuing commands from the R command prompt described below.

Subsection(Loading Tea in R)
When you start R (from the directory where the data is located), you are left at a
simple command prompt, $>$, waiting for your input. Tea extends R via a library of
functions for survey processing, but you will first need to load the library, with:

RCode(
library(Tea)
)

Now you have all of the usual commands from R, plus those from Tea. You only
need to load the library once per R session, but it's harmless if you run
tt(library(Tea)) several times.

After loading the library by running tt(library(tea)), you would then need to 
tell R to read your spec file, perform the checks, perform the imputations, 
and then finally check out the imputations to an R data structure so that you 
can view them. Observe the following code:

RCode(
library(tea)
readSpec("spec")
doChecks()
doMImpute()
checkOutImpute("fulldata")
)

These commands could be entered in a script file as easily as they are entered on R's command line.
You always have the option of creating a tt(.R) file that has each of the above commands listed 
sequentially. Observe the following example of a tt(.R) file:

RCode(
library(Tea)
library(ggplot2)
readSpec("demo.spec")
doChecks()
doMImpute()
checkOutImpute()
)

If we assume that the file above is named tt(demo.R) then we could run the following command 
from R's command line:

RCode(
source("demo.R")
)

Starting R and sourcing a script allows you to further query and interrogate the data
after execution. To run tt(demo.R) without starting R's interactive shell, use {\tt R -f
demo.R}.

In either case, once your spec file has been correctly implemented, your data will
be imputed and available for viewing through an R data frame.  We examine how this is
done in the next subsection.

Subsection(Showing the data)
Data is stored in two places: the database, and R data frames. Database tables
live on the hard drive and can easily be sent to colleagues or stored in
backups. R data frames are kept in R's memory, and so are easy to manipulate and
view, but are not to be considered permanent. Database tables can be as large as
the disk drive can hold; R data frames are held in memory and can clog up memory if
they are especially large.

You can use the tt(teaTable) function to pull a part of a database table into
an R data frame. You probably don't want to see the whole table, so there are
various options to limit what you get. Some examples:
RCode(
TeaTable("dc", cols="AGEP, PUMA", where="PUMA=104")
TeaTable("dc", cols="AGEP, PUMA", limit=30, offset=100)
)

The first example pulls two columns, but only where tt(PUMA=104). The second
example pulls 30 rows, but with an offset of 100 down from the top of the
table. You will probably be using the tt(limit) often; the tt(offset) allows
you to check the middle of the table, if you suspect that the top of the table
is not relevant or representative of the data need to analyze.

There are, of course many more options than those listed here. If you'd like to see what other tools are available in manipulating R data, you can get them via R's help system as such:

RCode(
?TeaTable
)

This tt(?name) form should give you a help page for any function, including Tea
functions like tt(?doRaking) or tt(?doMImpute). (Yes, Tea function documentation is still a
little hit-and-miss.) Depending on R's setup, this may start a paging program that lets 
you use the arrow keys and page up/page down keys to read what could be a long document. 
Quit the pager with tt(q).

The tt(show\_db\_table) function creates an R data frame, and, because the examples above didn't do
anything else with it, displays the frame to the screen and then throws it out. Alternatively, you can
save the frame and give it a name. R does assignment via tt($<$-), so name the output
with:

RCode(
p104 <- TeaTable("dc", cols="AGEP, PUMA", where="PUMA=104")
)

To display the data frame as is, just give its name at the command line.

RCode(
p104
)

But you may want to restrict it further, and R gives you rather extensive control over
which rows and columns you would like to see.

In both R and the spec file syntax, a tt(\#) indicates a comment to the human
reader; everything from the tt(\#) to the end of the line is ignored by the parser.

RCode(<|
p104[1,1]          #The upper-left element
p104[1,"AGEP"]     #The upper-left element, using the column name
p104[ ,"AGEP"]     #With no restriction on the rows, give all rows---the full AGEP vector
p104[17, ]         #All of row 17

minors <- p104[, "AGEP"] < 18
minors             #A true/false vector showing which rows are under 18.
p104[minors, ]     #You can use that list of true/falses to pick rows. This gives all rows under 18yo.
|>)

These commands could be entered on R's command line as easily as in a script file, so an
analyst who needs to verify the results of the consistency-checking step could copy and
paste the first three lines of the script onto the R command prompt, where they will run and
then return the analyst to the command prompt, where he or she could print subsections of
the output tables, check values, modify the spec files and re-run, continue to the
imputation step, et cetera.


Section(Basic specification syntax)
Everything in the spec file is a key/value pair (or, as may be more familiar to 
you, a tag: data definition). Each key in the spec file has a specific purpose and 
will be outlined in this tutorial.
To begin, we start in the header of tt(demo.spec):

SpecCode(
database: demo.db
id: SSN
)

Here, tt(database: demo.db) and tt(id: SSN) are examples of key: value pairs.
As is the case in tt(demo.spec), your spec file must begin by providing the
database (tt(database:your\_database.db)) and the unique identifier ({\tt
id:your\_unique\_id}). The tt(database) key identifies the database file
where all of your data will be manipulated during the various processes of Tea.
The tt(id) key provides a column in the database table that serves as the unique
identifier for each set of data points in your data set. Though the tt(id) key is
not strictly necessary, we strongly advise that you include one in your spec file to
prevent any unnecessary frustration as most of Tea's routines require its use. More
information on both of these keys can be found in the appendix of this tutorial.\\

You may have noticed that keys are assigned values with the following syntax:\\

tt(key: value)\\

This syntax is equivalent to the following form with curly braces:
SpecCode(
database{
	demo.db
}

id {
	SSN
}
)

Clearly, this form is not as convenient as the tt(key: value) form for single values. However, it allows 
us to have multiple values associated with a single line of data, and even subkeys. For example, take  
the next line in tt(demo.spec) (the computer will 
ignore everything after a \#, so those lines are comments for your fellow humans):

SpecCode(
checks { #all of the following values are associated with the "checks" key
	age < 15 && status='married'
	age > 99
}
)

The elements in a group don't just have to be values; they can also be key:value pairs:

SpecCode(
input { 
    input file: ss08pdc.CSV
    overwrite: no
    output table: dc
}
)

%  DV: Here we should explain how subkeys are used (a fact of which I'm not entirely 
%  sure myself). After explaining this we have:

We thus have a main key, tt(input) and subkeys.  This is useful for keeping the spec
file organized and making sure that keys from one part of the pipeline do not clash with
keys from other parts.  In the database, the keys and subkeys get merged together,
to produce a key:value table that, given the above inputs, would look like this:

\begin{verbatim}
key                 value
------------------  -----------
database            demo.db
id                  SSN
input/input file    ss08pdc.CSV
input/overwrite     no
input/output table  dc
\end{verbatim}

It is worth getting familiar with this internal form, because Tea prints error messages
using this form.  We will discuss this more later in the tutorial when we talk about
running your spec file in R.  Here is a summary of the syntax rules for keys:

    Items(
    ∙ You can have several values in curly braces, each on a separate line, which are
    added to the key/value list. Order is preserved.

    ∙ You can have as many levels of subkeys as you wish.

    ∙ If there is a subkey, then its name is merged with the parent key via a slash.

    ∙ As a shortcut, you can replace tt(key \{single value\}) with tt(key:single value).

    ∙ Each value takes up exactly one line (unless you explicitly combine lines; see
    below). Otherwise, white space is irrelevant to the parser. Because humans will also
    read the spec file, you are strongly encouraged to use indentation to indicate group
    membership.

    ∙ If you need to continue a value over several lines, put a backslash at the end
    of each line that continues to the next; the backslash and the newline following it
    will be replaced with a single space.

)

Paragraph(Two more tricks) There are two more conveniences to the spec file syntax,
which can be skipped on a first reading.

First, the special key tt(include) indicates that a separate file should be pasted in
where the tt(include) key was given. For example, we may have a file named tt(checks.spec):

SpecCode(
age < 15 && status='married'
age > 99
)

The main spec file would have a tt(checks) section that looks like this:

SpecCode(
checks{
    include: checks.spec
}
)

The tt(checks.spec) file would be pasted into place to reproduce the same tt(checks)
section written in one place above. This mechanism allows one person to work on the
consistency rules in one file and another to work on the imputations in another file.

Second, smaller segments can also be pasted into place. This was originally intended to
minimize redundancy in the tt(impute) segments below. The special tt(paste in) key
will paste in a given set of key:values at the point given. Thus, the spec file on the
left will be read as the one on the right:

\begin{minipage}{0.45\linewidth}
SpecCode(
cats{
    categories{
        age
        sex
        tract
    }


impute {
    paste in: cats
    output var: income
    method: lognormal
}

impute {
    paste in: cats
    output var: sex
    method: hot deck
}
)


\end{minipage}
{\Large $\Rightarrow$}\hspace{1cm}
\begin{minipage}{0.45\linewidth}
SpecCode(
impute {
    categories{
        age
        sex
        tract
    }
    output var: income
    method: lognormal
}

impute {
    categories{
        age
        sex
        tract
    }
    output var: sex
    method: hot deck
}
)
\end{minipage}

Notice that the key tt(cats) gets used by tt(paste in), but does not appear in
the final spec file.


Subsection(Input declarations)
We now continue through our spec file to the tt(input) key.

The tt(input) key specifies:
\begin{enumerate}
∙ The  CSV\footnote{The custom is to call these files {\em
    comma-separated values}, but you are not required to use commas as a delimiter.}
    file from which you will draw your data.
∙ The option to \texttt{overwrite} the CSV file currently written into the specified database with 
a new CSV file that you have specified.
∙ The tt(output table) key that specifies the table where you would like to 
write the data that is read in from the tt(CSV) file.
\end{enumerate}


You may have auxiliary data (such as administrative records) in a second table that need
to be merged with the main data set. Add a tt(join) group to the spec to make this
happen:

SpecCode(
join {
    host: dc
    add: auxtable
}
)

Items(
∙ You can have multiple tt(input) segments, should you need to read in multiple CSV
files.
∙ Both files must have a field with the name you specified in the tt(id) key in your
spec. There should be one row in each table for each ID.
)

\cmt{
More information about these keys can be found in the appendix.  A visual layout of
what the effects of the tt(input) segment of the spec affects is shown in Figure
Ref(inputfigure).

\begin{figure}
\begin{center}
\includegraphics[scale=.5]{dotInput.png}
\caption{input \{ $\ldots$ \} flow chart}\label{inputfigure}
\end{center}
\end{figure}
}

Subsection(Field declarations) The edit-checking system needs to know the type and range
of every variable that has an associated edit. If a variable does not have an associated
edit, there is no need to declare its range.

The declaration of the edit variables contained in the tt(fields) key consists of the field name, 
a type (tt(int), tt(cat), or 
tt(real)) to be discussed further below, and a list of valid values for integer and categorical variables. Here is a typical example:

SpecCode(
fields {
    age: int 0-100
    sex: cat M, F
    hh_type: int 1, 2, 4, 8
    income: real
}
)

The above code declared four variables: tt(age), tt(sex), tt(hh\_type), and
tt(income) and we've declared those four variables in different and valid ways.
By declaring our variables with a type and range, we can pass the information to the
edit-checking system so that it knows what to verify when running its checks for each
of these variables.  You can see that the list of values may be text categories or
numeric values, and the range 0--100 will be converted into the full sequence 0, 1,
2, \dots, 100.

When declaring a variable, the first word following the tt(:) must be a type. For instance, 
for the tt(hh\_type: int 1, 2, 4, 8) field above, we used the word 'int' to indicate that 
the data values of the field were of type integer. If leading zeros are relevant (e.g.,
a category like tt(01)), specify the field as a text category to prevent the leading
zero from being removed.  Keep in mind 
that if you declare the incorrect type for a field that the edit-checking system may not correctly 
verify the values of that field in your data set.

As a final note, we warn the user against creating fields with an overly large range of
integer values. For a field with a range of possible integer values, the edit-checking
system will verify that each data point falls into one of the possible values specified
in the range. Though this is easily doable for smaller ranges such as 0-100 or even
0-1000, it becomes extremely time consuming for larger ranges such as 0-600000. Instead,
we recommend assigning a field with a large range as a real variable.


Items(
∙ Any variable used for consistency checking needs to have a key:value line in the
tt(fields) section.
∙ The value is of one of three forms:
    Items(
        ∙ tt(int <values>)
        ∙ tt(cat <values>)
        ∙ tt(real)
    )
∙ The values for categories can be any list, separated by commas. Spaces before and
after commas are removed; spaces within category names are kept. 
∙ Integer values are a comma separated list of numbers and ranges, such as tt(-2, -1, 1-20, 20-30, 32).
)

%Here is a visual representation of the tt(fields) key:
%
%\begin{figure}[H]
%\begin{center}
%\includegraphics[scale=.5]{dotFields.png}
%\caption{fields \{ $\ldots$ \} key flow chart}\label{afigure}
%\end{center}
%\end{figure}

Subsection(Recodes)
Recode keys are new variables that are deterministic functions of 
existing fields. Figure Ref(basicrecode) presents a typical example.

\begin{figure}
SpecCode(
recodes {
    CATAGE {
        0 | age between 0 and 15
        1 | age between 16 and 64
        2 | age > 64
    }
}
)
\caption{A typical recode.}\label{basicrecode}
\end{figure}

The example declares a new variable CATAGE whose data points are based on a
deterministic function of the variable AGEP. As we will see later, this recode will
be called in the tt(categories) key during imputation so that the data points we
are attempting to impute will be imputed in categories based on their recode values
rather than collectively as a single set of data points.

Recodes are created via an SQL view, which is a virtual table that calculates the
values of recode fields on demand.

The name of the recode table is always the name of the input table preceded by {\tt
view}.  For example, the demo has an input table tt(dc), so subsequent steps use
the recode view tt(viewdc) as input.

Items(
∙ The parsing of recodes is a little different from the other parsing. We may change
this in the future. 
∙ Typical recodes have a name of the variable being created, followed by a sequence of
values of the form tt(<value> | <formula>).
∙ The tt(<value>) can be anything, but is typically a small integer.
∙ The tt(<formula>) is any SQL expression using existing variables; when the formula
is true, the variable takes on the associated value.
∙ If a recode does not have a $|$, then it is interpreted as a plain SQL formula for
defining the recode.
∙ There can be multiple tt(recodes) segments. They are executed in sequence, so
later recodes can make use of variables set up in earlier recodes.
)

\cmt{
Figure Ref(recodefigure) is a visual representation of the tt(recodes) key:

\begin{figure}
\begin{center}
\includegraphics[scale=.5]{dotRecodes.png}
\caption{records \{ $\ldots$ \} flow chart}\label{recodefigure}
\end{center}
\end{figure}
}


Subsection(Group recodes) Tea does all consistency checking on a single observation by
itself. Thus, to make an edit like {\em people with relationship status of child must
be younger than the householder} work, the age of the householder must be on the same line
as the child's age.

Group recodes are much like individual recodes: they are a new variable that is a function
of an existing variable (perhaps from an earlier recode group in the spec file), but the
expression for generating the variable will be evaluated using all of the data in a
household. 

Before discussing group recodes, here are a set of of single-record recodes
demonstrating the use of the SQL tt(case) statement,
which is analogous to the if-then-else expressions from many other languages. You will
rarely (if ever) need this in individual recodes via Tea, because the list-of-categories
syntax shown in Figure Ref(basicrecode) simplifies writing out the cases.

SpecCode(
recodes {
    #This syntax is for demonstration purposes only.
        MOVE: case MIG when 1 then 'moved' else 'stayed' end
        DEG: case when SCHL in ('24','23','22','21','20') then 'degree' else 'non-degreed' end
        MF: case SEX when '1' then 'Male' when '2' then 'Female' else NULL end
        REL: case RELP when '00' then 'Householder' when '01' then 'Spouse' \
                        when '02' then 'Child' \
                        when '06' then 'Parent' else NULL end
}
)

The group recodes in Figure Ref(morecodes) will produce the given variables
by applying the given formula to every person in the survey with the same tt(SERIALNO)
(one ACS household). Thus, each member of a household will have an tt(NP) field listing
the total number of people in the household, an tt(NHH) field listing the number of
people in the household who are householders, et cetera. 

This is where the tt(case) statement becomes necessary, because for some group recodes
we want to use only the information regarding a subset of household members.  The age of
householder is the maximum age in the household only among those with tt(RELP='00'),
which is expressed using a tt(case) statement inside of the tt(max()) function.

\begin{figure}
SpecCode(
group recodes {
    group id: SERIALNO
    recodes {
        NP: max(SPORDER)
        NHH: sum(RELP='00')
        NSP: sum(RELP='01')
        NUP: sum(RELP='15')
        HHAGE: max(case RELP when '00' then AGEP end)
        SPAGE: max(case RELP when '01' then AGEP end)
        SPORD: max(case RELP when '01' then SPORDER end)
        HHSEX: cast(round(avg(case RELP when '00' then SEX end)) as integer)
        SPSEX: cast(round(avg(case RELP when '01' then SEX end)) as integer)
    }
}
)
\caption{Group recodes}\label{morerecodes}
\end{figure}

Subsection(Checks)
The consistency-checking system is rather complex, but this complexity is what makes
the system efficient and reliable: there are typically a few dozen to a few hundred
checks that every observation must pass, from sanity checks like em(fail if age < 0)
to real-world constraints like em(fail if age < 16 and status='married'). Further,
every time a change is made (such as by the imputation system) we need to re-check that
the new value does not fail checks. For example, an OLS imputation of age could easily
generate negative age values, so the consistency checks are essential for verifying that
the imputation process is giving us accurate and usable data points. In addition to
error checking we can also use consistency checks for other purposes, such as setting
em(structural zeros) for the raking procedure.

All of the checks that are to be performed are specified in the tt(checks) key. 
The checks you specify here will be performed on all input variables, as well as on all 
imputed data values. Let's take a look at an example tt(checks) key:

SpecCode(
checks {
    	age < 0
        age > 95 => age = 95
       }
)

This indicates that the consistency checking system should verify that age is not less
than 0 and that age is not greater than 95. Additionally we specify that a value higher
than 95 should simply be top-coded as 95. We have not included an automatic fix for
age < 0 because if a data-point has a negative age value than it's indicative of a
processing error in the incoming data.

\cmt{
Figure Ref(checkflow) is a visual representation of the tt(checks) key and all of its subkeys.

\begin{figure}
\begin{center}
\includegraphics[scale=.5]{dotChecks.png}
\caption{checks \{ ... \} key flow chart}\label{checkflow}
\end{center}
\end{figure}
}


Subsection(Imputation) \label{imputesec}
The tt(impute) key has several sub-keys that describe various steps in the imputation
process. Many of the values of these sub-keys are derived from values found earlier
in the spec file, e.g. tt(categories) is based off of the variables declared in {\tt
recodes}. Here is an example of an tt(impute) key that is used to
outline the imputation of the age variable described above:

SpecCode(
impute {
    input table: viewdc
    min group size: 3
    draw count: 3
    seed: 2332

    categories {
        CATAGE
        SEX
    }

    method: hot deck
    output vars: age
}
)

Let's walk through each of the sub-keys above and see what they're doing:
Items(
∙ The tt(output table) sub-key specifies the name of a view table to 
where you will write your recode variables. 
∙ tt(min group size)\ indicates the minimum number of known data points that can be 
used as donor or training data to impute unknown data points. 
∙ tt(draw count)\ determines the number of multiple imputations that should be performed. 
Because imputation can be done using stochastic models, it is often beneficial to obtain multiple 
imputations of the same data set. This will be explained in more detail later. 
∙ tt(seed)\ specifies the pseudo-random number generator seed that is used to obtain 
random numbers for stochastic imputation models. Running Tea with the same seed will
always produce identical results, so it is good form to record the seed as part of the 
specification. Changing the random seed will produce a new stream of random draws.
∙ tt(categories)\  specifies which recodes will be used to impute the data set for 
the given variable. For example, if tt(CATAGE) is listed in the tt(categories) key then the 
data points that correspond to each of the three possible tt(CATAGE) values will be imputed 
separately. While it's not an absolute requirement that the tt(categories) key be implemented, most data 
sets are imputed by categories so you will be using this key for almost all of your imputations.
∙ tt(method)\ specifies the type of model that is used to impute your data points. 
Choosing this model is discussed more thoroughly later on in the tutorial.
∙ tt(output vars)\ specifies the variable whose data points are to be imputed. For this reason 
you could consider tt(output vars) as both the `input' variable as well as the `output' variable 
to the model specified in tt(method).
)
More can be found on each of the above keys in the appendix.

\cmt{
Figure Ref(imputefigure) is a visual representation of the tt(impute) key how the
elements are used.

\begin{figure}
\begin{center}
\includegraphics[scale=.5]{dotImpute.png}
\caption{impute \{ ... \} key flow chart.}\label{imputefigure}
\end{center}
\end{figure}
}

Section(Flagging for disclosure) For a given crosstab, there may be cells with a
small number of people, perhaps only one. This feature of Tea allows us to flag those
cells for later handling.

Any combination of variables could be a crosstab to be checked, but flagging
typically focuses on only a few sensitive sets of variables. Here is the section
of the spec file describing the flagging. The tt(key) list gives the variables
that will be crossed together. With tt(combinations: 3), every set of three
variables in the list will be tried. The tt(frequency) variable indicates that
cells with two or fewer observations will be marked.

We are calling this specific form of disclosure avoidance em(fingerprinting), so after
this segment of the spec file is in place, call em(doFingerprinting()) from R to run the
procedure. The output is currently in a database table named em(vflags).

SpecCode(
fingerprint {
    key{
        CATAGE
        SEX
        PUMA
        HISPF
        ESR
        PWGTP
    }
    frequency: 2
    combinations: 3
}
)

The fingerprinting algorithm determines which fields are sufficient to eliminate
disclosure risk for a given record, and then blanks those fields.


Thus, by the end of the edit and disclosure-avoidance steps,
a record in a data set might have several fields which require imputation for different reasons:
Items(
∙ A record could have several inconsistent items that we must replace.
∙ An otherwise consistent record could have a blank item that we wish to impute.
∙ A record could have a combination of consistent items that could lead to personal identification.
)

The imputation routines in Tea find values to fill in all of these holes.

\cmt{
Each scenario implies slightly different knowledge about the data, and thus each scenario
might require a different imputation method to properly use this knowledge.

An overlay is a secondary data table (or set of tables) that gives information regarding
the \emph{reason} for imputation.  Using missing data as an example, a simple overlay
could have an entry for each item in the data, indicating if that item is missing or not.
A more complicated overlay could delineate the type of non-response for each data item.
}

Section(Imputation)\label{impsec}
Thus far we've seen how to impute a data set using a single tt(impute) group. In this form, single 
imputation produces a list of replacement values for those data items that fail consistency checks 
or initially had no values. For the case of editing, the replacements would be for data that fails 
consistency checks; for the case of basic imputation, the replacements would be for elements that
initially had no values. Recall that as we stipulated earlier in the tutorial, for our purposes we 
consider a looser definition of imputation that includes the replacement of data points that both 
initially had no value as well as those that fail consistency checks.

In a similar vein, while single imputation gives us a single list of replacement values, any stochastic 
imputation method could be repeated to produce multiple lists of replacement values. For instance, 
we could use multiple imputation to calculate the variance of a given statistic, such as average 
income for a subgroup, as the sum of within-imputation variance and across-imputation variance.

To give a concrete example, consider a randomized hot deck routine, where missing values are
filled in by pulling values from similar records. For each record with a missing income:
Items(
TeX(\tighten)
∙ Find the universe of which the record is a member.
∙ For each variable in turn:
    Items(
    ∙ Make a draw from model chosen for the variable, based on the specific universe from (1).
    )
)


The simplest and most common example is the randomized hot deck, in which each survey respondent 
has a universe of other respondents whose data can be used to fill in the respondent's missing
values. The hot deck model is a simple random draw from the given universe for the given variable.

Given this framework, there are a wealth of means by which universes are formed, and
a wealth of models by which a missing value can be filled in using the data in the chosen universe.

Paragraph(Universes)
The various models described above are typically fit not for the whole data set, but for
smaller em(universes), such as a single county, or all males in a given age group.
A universe definition is an assertion that the variable set for the records in the
universe was generated in a different manner than the variables for other universes.

An assertion that two universes have different generative processes could be construed in
several ways:

Items(TeX(\tighten)
∙ different mathematical forms, like CART vs.\ logistic regression
∙ One broad form, like logistic regression, but with different variables chosen
    (that is, the stochastic form is the same but the structural form differs).
∙ A unified form, like a logistic regression with age, sex, and ancestry
    as predictors, with different parameter estimates in each universe.
)

Universe definitions play a central role in the current ACS edit and imputation system.
Here, universes allow data analysts to more easily specify particular courses of
action in the case of missing or edit-inconsistent data items. To give an extreme example, for the imputation of marital status in ACS group quarters (2008), respondents are placed
in two major universes: less than 15 years of age (U1) and 15 or more years of age
(U2). The assertion here, thus, is that people older than 15 have a marital status
that comes from a different generative process than those people younger than 15.
This is true: people younger than 15 years of age cannot be married!  Thus in the
system, any missing value of marital status in U1 can be set to ``never married'',
and missing values in U2 can be allocated via the ACS hot-deck imputation system.

Now that we are more familiar with universes, we can discuss the various models that are 
available in Tea and the method of choosing the one that is appropriate for the 
imputation being performed.


Subsection(Models)
Now that we're more familiar with the concept of universes, we examine how to choose the 
model that will give us the best results when imputing the data points in a specific universe.
Indeed, given an observation with a missing data point and a universe, however specified, there is
the problem of using the universe to fill in a value. Randomized hot-deck is again
the simplest model: simply randomly select an observation from the universe of acceptable
values and fill in. Other models make a stronger effort to find a somehow-optimal value:\\

$\bullet$ One could find the nearest neighbor to the data point to be imputed (using any
of a number of metrics).\\

$\bullet$ Income is typically log-Normally distributed, and log of this year's income, last
year's income, and if available, income reports from administrative records (\adrec) may
be modeled as Multivariate Normal (with a high correlation
coefficient among variables). After estimating the appropriate Multivariate
distribution for the universe, one could make draws from the distribution.\\

$\bullet$ If the Multivariate Normal seems implausible, one could simply aggregate the
data into an empirical multivariate PDF, then smooth the data into a kernel
density estimator (KDE). Draws from a specified KDE can be made as easily as
draws from a standard Multivariate Normal.\\

$\bullet$ Count data, such as the number of hospital visits over a given period, are
typically modeled as a Poisson distribution. In a manner similar to fitting a Normal, one
could find the best-fitting Poisson distribution and draw from that distribution to fill
in the missing observation.\\

$\bullet$ Bayesian methods: if \adrec are available, they may be used to generate a prior distribution
on a variable, which is then updated using non-missing data from the survey.\\

$\bullet$ One could do a simple regression using the data on hand. For example,
within the universe, regress income on  age, race, sex, and covariates from \adrec.
Once the regression coefficients are calculated, use them to impute income for
the record as the predicted value plus an error term. \cmt{ Note that OLS assumes
a Normally-distributed error term and therefore gives a distribution of values for
the predicted income, not just a single number; therefore one could draw multiple
imputations.}\\

$\bullet$ Discrete-outcome models, like the Probit or Logit, could be used in a similar
manner.\\

$\bullet$ To expand upon running a single regression, one can conduct a structured search
over regression models for the best-fitting regression.\footnote{Because the imputation
step is not an inference step, such a search presents few conceptual difficulties.}
Such a search is currently in use for disclosure avoidance in ACS group quarters.\\

A unified framework would allow comparison across the various imputation
schemes and structured tests of the relative merits of each. Though different surveys
are likely to use different models for step (2) of the above process, the
remainder of the overall routine would not need to be rewritten across surveys.

We now discuss each of the models that are available in Tea.

\cmt{DV: We need to add rel to this list.}
Subsection(Models of Tea) This section describes the models currently available from
Tea in greater detail.

Paragraph(Hot deck) This is randomized hot deck. Missing values are filled in by
drawing from nonmissing values in the same subset. The assumption underlying the model
is em(missing completely at random) (MCAR), basically meaning that nonrespondents
do not differ in a systematic way from respondents.

This model has no additional keys or options, although the user will probably want an
extensive set of category subsets. Example:

SpecCode(
common spec {
    input table: dc
    min group size: 3
    draw count: 5
    id: serialno
   categories {
        agecat
        sex
    }
}

impute{
      # Here, Tea will detect that sex is used both as a category and 
      # the target variable, and will not use it as a category.
    paste in: common spec
    output vars: sex
    method: hot deck
}

impute{
    paste in: common spec
    method: ols
    output vars: wagp
    input vars: rac1p, agep, nativity||sex
}
)

Paragraph(Ordinary least squares (aka regression))  This is the familiar $y = \beta_0
+ \beta_1 x_1  + \beta_2 x_2 + \dots +\epsilon$ form. The parameters (including the
variance of $\epsilon$) are estimated for the subset of the category where all variables
used in the regression are not missing. For a point where $y$ is missing but $(x_1, x_2, \dots)$ are not,find $\beta_1 x_1  + \beta_2 x_2 + \dots$, then add a random draw from the distribution of $\epsilon$.

The variables may be specified via the usual SQL, with two exceptions to accommodate the
fact that so many survey variables are categorical.

\cmt{%BK: Not sure how much of this still works.
Unless otherwise noted, all dependent variables are taken to be categorical, and so are expanded to
a set of dummies. The first category is taken to be the num\'eraire, and others are broken
down into dummy variables that each have a separate term in the regression. The
independent variable will always be calculated as a real number, but depending on the type
of variable may be rounded to an integer.

If a dependent variable is numeric, list it with a \#, such as {\tt variables:
\#age, sex}.

An em(interaction) term is the product of the variables, where for categories {\em
product} means the smaller subsets generated by the cross of the two variables, such as
a sex-age cross of $(M, 0--18), (F, 0--18), (M, 18--35), (F, 18--35)$; for continuous
variables em(product) has its typical meaning.
}

\cmt{
Because every dependent variable must be complete for the regression to calculate the
independent variable, it makes sense to put regressions in a sequence, with a spec file like:

SpecCode(
impute [first]{
        models{
        rac1p { method: hot deck }
        nativity { method: hot deck }
        sex { method: hot deck }
        }
}

impute [second]{
        models{
        agep { method: ols
               vars:  rac1p, nativity||sex
        }
        }
}
)

and then two steps in R:

RCode(
doMImpute(first)
doMImpute(second)
)
}

Paragraph(Probit and Logit) These models behave like OLS, but are aimed at categorical
variables. The output variable is not restricted to two categories.

%Need to fill in this paragraph
%Paragraph(seqRegAIC)

Paragraph(Distributions) The Normal (aka Gaussian) distribution is the archetype of this
type of model. First, estimate the mean and standard deviation using the non-missing data
in the category. Then fill in the missing data via random draws from the Normal
distribution with the calculated parameters. You may use either tt(method: normal) or
tt(method: gaussian).

For other situations, other distributions may be preferable. For example, income is
typically modeled via tt(method: lognormal). Count data may best be modeled via {\tt
method: poisson}.

Hot deck is actually a fitting of the Multinomial distribution, in which each bin has
elements in proportion to that observed in the data; tt(method: hot deck) and {\tt
method: multinomial} are synonyms.

The em(method: multivariate normal) doesn't work yet.

The distribution models have no additional options or keys.

Paragraph(Kernel smoothing) A kernel density estimate overlays a Normal distribution over
every data point. For example, if a data set consisted of two data points at 10 and one at
12, then there would be a large hump in the final distribution at 10, a smaller hump at
12, and the distribution would be continuous over the entire space of reals.

Thus, kernel smoothing will turn a discrete distribution consisting of values on a
few values into a continuous distribution.

Invoke this model using either tt(method: kernel) or tt(method: kernel density).

\todo{We have written but not incorporated an HTML viewer for data using overlay
information.}

\cmt{
Section(Synthesis)
Group quarters data in the 2010 Decennial census will be protected via
a method known as ``synthetic data''. This method uses statistical models to modify
records in need of confidentiality protection. Such a method has already been applied to
group quarters records in the American Community Survey for collection years 2006--2009.

Section(Weighting)
We use the em(survey) package for R to calculate and modify weights for a
survey. The package is fully documented in \citet{lumley:surveys}.

Section(Models and Universes)
Statistical models are, in essence, an assertion about the \emph{process} that generates
the data. As such, useful models for imputation are those that imply processes that
we believe generate ACS GQ data items. Models are typically prescribed at the level
of variables; that is, we seek to describe the generative process of a set of variables
for a certain population.

A given variable set could have several distinct processes by which its values
are generated. In this case, it would be suboptimal to use only a single model to
perform imputation for this variable set. We group respondent records together into a
``universe'' when we believe their values for a given set of variables are generated
by the same process. Different sets of variables that require imputation could thus
require different universe definitions.

Subsection(Models)
Here, we list several models used to predict or fill in a missing element of
a record. The element could be missing because it was not reported (imputation),
because it was incorrectly reported (editing), or because leaving the correct value
as it lays would create disclosure risk. For any of these cases, any of the models to
follow could be applied. Note that these models are typically inserted into a larger
procedure; for example, a series of models could be used to fill in various data items.

{\bf Note on MI: to be ``proper'' MI models, they would need to produce
posterior predictive densities, so we might mention that deterministic
procedures don't fit the bill}

\subsubsection{Randomized hot deck/multinomial} This is a univariate method, that
randomly draws a value from the list of nonmissing values for the variable.
{\bf you can have a multivariate hot deck: we could get a value of A and B
from a cell defined by C and D}

\subsubsection{Nearest-neighbor} Based upon some metric defined by the user, find the
record closest to the record to be filled in, and copy the value(s) from that
record.

\subsubsection{Probabilistic nearest-neighbor} Define the odds of selecting a record
as inversely proportional to the record's distance to the record to be filled
in. That is, the nearest neighbor is the most likely to be selected, but others
may also be selected. After selecting a record, copy the value(s) as with
the deterministic nearest-neighbor model.

\subsubsection{Linear models} Given a complete-data subset, fit a generalized linear
model (GLM) to be specified by the user. The predictors used for a model could
be selected by hand or could be chosen automatically based on a given
criterion, such as the Akaike/Bayesian Information Criterion.
{\bf we could also suggest sequential models that start of predicting based
upon complete items; that is, if V1 is complete in the subset, first do
V2 \~{} V1, then V3 \~{} V1 + V2, etc; if nothing in complete, we could
always pick one to fill in randomly (that is, just use a prior)}

}

Recall that we had briefly discussed multiple imputation in the tt(impute) section above. 
We now discuss this in more detail.

Subsection(Multiple Imputation)
A single imputation would produce a list of replacement
values for certain data items. Any stochastic imputation method could be repeated
to produce multiple lists of replacement values. Variance of a given statistic,
such as average income for a subgroup, is then the sum of within-imputation variance and
across-imputation variance.

Items(
TeX(\tighten)
    ∙ For each imputation:
        Items(
        TeX(\tighten)
            ∙ fill in the data set using the given set of imputed values
            ∙ calculate the within-imputation variance
        )
    ∙ Sum the across-imputation and average within-imputation variances to
    produce an overall variance figure that takes into account uncertainty due to imputation.
)

The question of what should be reported to the public from a sequence of imputations
remains open. The more extensive option would be to provide multiple data sets; the less
extensive option would be to simply provide a variance measure for each data point that
is not a direct observation.




Section(Synthesizing microdata) 

SpecCode(
raking {
    all_vars: puma|catage|rac1p

    contrasts{
        catage | rac1p
        puma | rac1p
        puma | catage
    }
}
)

\cmt{
As you have seen a few times to this point, once you have the spec file in place you can
call the procedure with one R function, which in this case is tt(doRaking()). But there
are several ways to change the settings as you go, so that you can experiment and adjust.

The first is to simply change the spec file and re-run. To do this, you will have to call
tt(read\_spec) again:
RCode(
read_spec("demo.spec"); doRaking()    #Run two commands on a line by ending the first with a semicolon
read_spec("demo.spec"); doRaking()    #Hit the up-arrow to call up the previous line.
)

Everything you can put in a spec file you can put on the command line. The help system
will give you the list of inputs (which will also help with writing a spec file), and you
can use those to tweak settings on the command line:
RCode(
?doRaking                   #What settings are available?
doRaking(max_iterations=2)  #What happens when the algorithm doesn't have time to
converge?
)
} %end commented-out raking section.





\cmt{
	DV: The two paragraphs below (and more specifically the figures in them) probably need 
	to be cleaned up. I got rid of the \listing{ ... } tags below that were around the 
	\lstlisting{...} tags because they weren't getting compiled correctly but if you 
	want to put them in the old code that originally had them can be found in 
	/Tea/doc/archived_tutorial_documents/tutorial.tex

Maybe usable as an example later.

        Paragraph(Interface) Figure Ref(sippconfig) shows a (slightly abbreviated)
        configuration file describing the hot deck process for a variable in the SIPP.
        It is intended to be reasonably readable, and maintainable by an analyst who is
        a statistician but not a programmer.

        Lines 11--18 of the configuration specify the
        categories used for step (1) above: draws are made from the universe of records with an
        age in the same age category as the record to be filled in and tt(num\_sipp\_jobs\_2008) in the same
        category as well.

        Of course, different surveys would have different classification schemes, but
        this means that each survey would need a new configuration file, not new code.

        Line eight indicates that five imputations are to be done for each missing
        value. Those with extensive experience with multiple imputation often advise
        that a handful of imputations are sufficient for most purposes.

        The sample from
        Figure Ref(sippconfig) focused on the determination of categories in which to
        do imputation. Figure Ref(acsconfig) focuses on regression models that go
        beyond the simple randomized hot deck of Figure Ref(sippconfig). Lines 5--8
        specify the variables that need imputation, and the form of model to be used.
        The current system will search the set of models of the given form for the one
        that best fits the known data; Lines 9--14 show the list of variables that could be
        used as explanatory variables, although a typical model will likely wind up
        using only around four or five.

     Paragraph(Edits) The system as written includes a component that checks consistency
        against a sequence of edits. In line three of the sample spec file of Figure
        Ref(sippconfig), the tt(flagearn) variable is declared to have possible values of 0,
        1, 3, or 4, but line four specifies that if an imputation returns 4, then it is rejected.
        The imputation routine sketched above does not need to include any edit rules, because
        this edit step will take those into account; the separation of edits and imputations
        simplifies the routine.
        \begin{lstlisting}[language=,numbers=left,numberstyle=\scshape]
        database: sipp.db

        |flagearn 0, 1, 3, 4
        flagearn = 4

        impute_by_groups {
            min_group_size  {20}
            iteration_count  {5}
            datatab  {sippdata}

            categories {
               15<=agesipp200812<18;
               18<=agesipp200812<22;
               22<=agesipp200812<40;
               40<=agesipp200812<62;
               62<=agesipp200812;
               num_sipp_jobs_2008 = 0;
               num_sipp_jobs_2008 = 1;
               num_sipp_jobs_2008 => 2;
         }

            imputes{
               flagearn~ flagearn;
            };
        }
        \end{lstlisting}

SpecCode(
        database: acs2008.db

        impute{
            seqRegAIC{
                vars{
                  TI{ model: gam }
                    DIS{ model: multinom }
                }
                predlist{ #Sample of predictors that could used for regressions
                    SEX; YOE; WKL; MIL; UR;
                    SCH; RCGP; POV; SS; MAR;
                    LANX; JWTR; TYPGRP; GQINST; FER;
                    ESR; DIS; COW; CIT; OCC2;
                }
            }
        }

)
}

Section(Conclusion)
This concludes our tutorial. If you have any questions or comments please feel
free to contact Rolando Rodr\'iguez (rolando.a.rodriguez@census.gov) or
Ben Klemens (ben.kle\-mens@census.gov). Also, check out the R-Forge Website at
\url{https://r-forge.r-project.org/projects/Tea/} for updates to both the Tea software
as well as this tutorial document.

\section*{Appendix: keys}

This is a reference list of all of the available keys that could appear in a spec file.
As a reference, descriptions are brief and assume you already know the narrative of the
relevant procedures, in the main text.

Keys are listed using the tt(group/key/value) notation described in Section Ref(basicssec).

%\input ../keys.tex


)
