---
title: "Quick Start"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Quick Start}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", fig.width = 7, fig.height = 5)
library(dsld)
set.seed(2026)
```

# Data Science Looks at Discrimination

DSLD provides statistical, graphical, and tabular methods for exploring possible
discrimination and bias. It supports two related questions:

- **Adjusted group comparisons:** how does an outcome differ between groups
  after accounting for selected confounders? For example, how do expected wages
  differ by gender among people with similar age, education, and occupation?
- **Fair prediction:** how can a model predict an outcome while limiting the
  influence of sensitive attributes and their proxies?

We use **Y** for the outcome, **S** for a sensitive attribute, and **X** for other
predictors. Many functions wrap base R, qeML, or fairml methods and add group
comparisons or fairness diagnostics. Their first arguments usually identify the
data frame, outcome column (`yName`), and sensitive column (`sName` or `sNames`).
Use `summary()` and `predict()` for the supported fitted-model classes.

# Analytical operations

## Group comparisons with confounder adjustment

The `lsa` dataset, supplied by qeML, contains law-school admissions and outcome
variables. We use LSAT score as Y, race as S, and the remaining columns as
covariates. Some covariates are
measured after the LSAT, so this is an illustration of the software rather than
a causal model of test-score differences.

```{r law-school-model}
data(lsa, package = "qeML")
names(lsa)
lsat_model <- dsldLinear(lsa, "lsat", "race1")
summary(lsat_model)
```

The summary reports model coefficients and pairwise sensitive-group differences,
including their standard errors. Factor columns are represented using the
fitted model's contrasts. For two groups, the comparison is the second factor
level minus the first; for more groups, the labels state each difference's
direction.

For example, the family-income coefficient describes the adjusted change in
expected LSAT score per income-category step. The race coefficients and group
comparisons describe adjusted associations under this model. Their practical
size and uncertainty matter alongside their p-values; these associations alone
do not establish discrimination or its cause.

Setting `interactions = TRUE` fits a separate model for each sensitive group.
Supply `sComparisonPts` to compare these fitted models at specific predictor
values. The repository README demonstrates this with wage income and age.
`useSandwich = TRUE` requests a sandwich covariance estimate for linear models.

## Fair prediction

A predictor can carry information about S even when S is omitted from the
model. Fair-prediction methods limit this influence through constraints or
feature deweighting. Evaluate prediction error and fairness diagnostics together:
a stronger constraint does not guarantee improvement in every measure.

The bundled `compas1` data include recidivism outcomes and demographic variables.
We illustrate `dsldFgrrm()`, a wrapper for fairml's generalized ridge method,
using a reproducible subset of two racial groups. `unfairness` lies in (0, 1];
smaller values impose a stronger fairness constraint under the selected
definition. Always identify the positive outcome using `yesYVal`.

```{r fair-models}
data(compas1, package = "dsld")
records <- droplevels(subset(compas1, race %in% c("African-American", "Caucasian")))
records <- records[, c("age", "priors_count", "sex", "two_year_recid", "race")]
records <- records[sample.int(nrow(records), 800), ]
train_rows <- sample.int(nrow(records), 600)
training <- records[train_rows, ]
testing <- records[-train_rows, ]
test_x <- testing[, setdiff(names(testing), "two_year_recid")]

baseline <- dsldFgrrm(training, "two_year_recid", "race", unfairness = 1,
                      yesYVal = "Yes")
constrained <- dsldFgrrm(training, "two_year_recid", "race", unfairness = 0.1,
                         yesYVal = "Yes")
baseline_result <- predict(baseline, test_x)
constrained_result <- predict(constrained, test_x)
data.frame(
  model = c("Baseline", "Constrained"),
  test_error = c(
    mean((baseline_result$preds > 0.5) != (testing$two_year_recid == "Yes")),
    mean((constrained_result$preds > 0.5) != (testing$two_year_recid == "Yes"))
  )
)
constrained_result$correlations
```

These probabilities describe the specified positive outcome. Correlations
measure association with sensitive attributes; their absolute magnitudes are
useful for comparison, but zero correlation is not a general fairness guarantee.
The historical fields `trainAcc` and `testAcc` contain **errors**, not accuracy:
mean absolute error for regression and misclassification rate for classification.

# Graphical and tabular operations

## Conditional disparity

`dsldConditDisparity()` plots an outcome against a numeric predictor, separately
by sensitive group. It can restrict rows through `condits` and omit groups with
fewer than `minS` rows. The binary plotting interface uses the first outcome
factor level as its positive class.

```{r conditional-disparity}
law_sample <- lsa[sample.int(nrow(lsa), 1500), ]
levels(law_sample$bar)
dsldConditDisparity(law_sample, "bar", "race1", "lsat", minS = 50)
```

Inspect the outcome coding, group sample sizes, and uncertainty before drawing
substantive conclusions from the curves.

## Group frequencies and possible confounders

The bundled `svcensus` data describe workers in engineering occupations in
Silicon Valley. Compare education and occupation distributions by gender:

```{r group-frequencies}
data(svcensus, package = "dsld")
names(svcensus)
dsldFrequencyByS(svcensus, cName = "educ", sName = "gender")
dsldFrequencyByS(svcensus, cName = "occ", sName = "gender")
tapply(svcensus$wageinc, svcensus$occ, mean)
```

Columns follow the factor levels shown in the table labels. Education codes 14
and 16 represent master's and doctoral degrees. Differences in occupation
frequencies and mean wages can motivate examining occupation as a possible
confounder. Selecting adjustment variables still requires substantive reasoning;
similar group frequencies alone do not rule a variable in or out.

`dsldDensityByS()` compares numeric distributions. `dsldConfounders()` applies
frequency or density comparisons to the other columns in a data frame.

## Frequency-based parallel coordinates

Parallel coordinates place each variable on a separate vertical axis and join
the values belonging to one observation. `dsldFreqPCoord()` can display the
most typical patterns within each sensitive group, using nearest-neighbor
density estimates. The axes are centered and scaled.

```{r parallel-coordinates}
plot_data <- law_sample[, c("fam_inc", "ugpa", "gender", "lsat", "race1")]
dsldFreqPCoord(plot_data, m = 25, sName = "race1", k = 15)
```

This view can reveal multivariate patterns that separate plots miss. Its density
and sampling settings affect which observations appear; it does not summarize
every member of a group.

## Comparing feature sets

`dsldTakeALookAround()` compares predictor subsets C. It reports:

- **a:** prediction error for Y using C.
- **b:** prediction error for Y using C and S.
- **c:** prediction error for S using C.

The same held-out rows are used for all feature sets. Lower (c) means S is more
predictable from C, which can indicate stronger proxy information. Interpret
these errors relative to suitable outcome baselines.

```{r feature-sets}
wages <- svcensus[sample.int(nrow(svcensus), 800),
                   c("age", "educ", "wkswrkd", "gender", "wageinc")]
dsldTakeALookAround(wages, "wageinc", "gender", maxFeatureSetSize = 2, holdout = 100)
```

# Python interface

Install the R package first, then run `python -m pip install ./inst` from the
repository root. The [Python guide](https://github.com/matloff/dsld/blob/master/inst/README.md)
includes installation instructions and three notebooks covering data conversion,
analysis, fair prediction, and graphics.

```python
import rpy2.robjects as ro
from rpy2.robjects.packages import importr
from dsldPy import dsldPyLinear, dsldPyLinearSummary

bundled = ro.Environment()
importr("utils").data("svcensus", package="dsld", envir=bundled)
model = dsldPyLinear(bundled["svcensus"], "wageinc", "gender")
dsldPyLinearSummary(model)
```

# Associated textbook and function reference

The [free companion textbook](https://matloff.github.io/dsldBook/)
explains the statistical concepts using DSLD examples. For the categorized
function reference, run `vignette("Function_List", package = "dsld")`.