{
  "alt": "Two tabs of line charts over the number of families. The first tab shows the silhouette of the tract data and of simulated data with no clusters. The two lines are close at two families, the tract line is much higher at four families, and the lines stay near each other for larger numbers. The second tab shows the agreement of refits, high up to five families, lower at six and lowest at seven.",
  "caption": "In the first tab, the tract line sits above the simulated line most clearly at four families, where the two are farthest apart; at two families the lines nearly meet. The tract line plots the mean silhouette of a seeded sample of tracts for each number of families, and the simulated line plots the same fit to clouds that keep each measure's distribution and the correlations among measures but have no clusters. In the second tab, agreement of refits with the fit on all tracts is high up to five families, lower at six and lowest at seven.",
  "data": [],
  "id": "number_of_families",
  "notes": [
    "The silhouette uses a seeded random sample of tracts; the intervals show its sampling error, at most {fact:sil_half_max} in either direction. The simulated line is the mean of {fact:n_null} draws that differ by at most {fact:null_draw_gap} at any number of families.",
    "Simulated data are drawn from a Gaussian copula: each measure keeps its distribution in the tract data, and the dependence among measures is a single multivariate normal cloud, so k-means has no clusters to find.",
    "The band in the second tab runs from the tenth to the ninetieth percentile of the refits, each a k-means fit on a resample that draws whole counties with replacement and assigns every tract to its nearest center (replicates and seed in the methods note)."
  ],
  "source": "ACS 2019-2023 deprivation measures, k-means fits to all tracts, silhouette on a sample of tracts; unweighted (each tract counts once)",
  "spec": {
    "panels": [
      {
        "figure": {
          "alt": "Two lines of silhouette against the number of families. The lines nearly meet at two families, separate widely at four families, and stay close for larger numbers.",
          "caption": "The gap between the lines is the evidence for clusters. It is widest at four families.",
          "data": [
            {
              "conf_high": 0.220602,
              "conf_low": 0.215555,
              "k": 2,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.218078
            },
            {
              "conf_high": 0.208102,
              "conf_low": 0.205354,
              "k": 2,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.206728
            },
            {
              "conf_high": 0.213586,
              "conf_low": 0.206821,
              "k": 3,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.210203
            },
            {
              "conf_high": 0.181924,
              "conf_low": 0.178618,
              "k": 3,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.180271
            },
            {
              "conf_high": 0.212123,
              "conf_low": 0.205923,
              "k": 4,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.209023
            },
            {
              "conf_high": 0.122189,
              "conf_low": 0.119515,
              "k": 4,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.120852
            },
            {
              "conf_high": 0.152951,
              "conf_low": 0.148087,
              "k": 5,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.150519
            },
            {
              "conf_high": 0.125934,
              "conf_low": 0.123012,
              "k": 5,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.124473
            },
            {
              "conf_high": 0.15575,
              "conf_low": 0.151057,
              "k": 6,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.153403
            },
            {
              "conf_high": 0.126815,
              "conf_low": 0.123975,
              "k": 6,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.125395
            },
            {
              "conf_high": 0.130576,
              "conf_low": 0.126051,
              "k": 7,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.128313
            },
            {
              "conf_high": 0.125492,
              "conf_low": 0.122767,
              "k": 7,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.12413
            },
            {
              "conf_high": 0.13427,
              "conf_low": 0.12941,
              "k": 8,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.13184
            },
            {
              "conf_high": 0.122318,
              "conf_low": 0.119705,
              "k": 8,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.121011
            },
            {
              "conf_high": 0.139795,
              "conf_low": 0.134812,
              "k": 9,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.137303
            },
            {
              "conf_high": 0.112601,
              "conf_low": 0.110078,
              "k": 9,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.111339
            },
            {
              "conf_high": 0.142469,
              "conf_low": 0.137453,
              "k": 10,
              "n": 10000,
              "series": "Tract data",
              "silhouette": 0.139961
            },
            {
              "conf_high": 0.111613,
              "conf_low": 0.109104,
              "k": 10,
              "n": 10000,
              "series": "Simulated, no clusters",
              "silhouette": 0.110358
            }
          ],
          "id": "number_of_families_silhouette",
          "spec": {
            "annotations": [
              {
                "label": "Four families",
                "type": "x-rule",
                "x": 4
              }
            ],
            "colors": {
              "Simulated, no clusters": "muted",
              "Tract data": "total"
            },
            "hi": "conf_high",
            "lo": "conf_low",
            "n": "n",
            "series": "series",
            "x": "k",
            "xDomain": [
              2,
              10
            ],
            "xLabel": "Number of families",
            "xTicks": "data",
            "y": "silhouette",
            "yDomain": [
              0,
              0.3
            ],
            "yFormat": {
              "digits": 2
            },
            "yLabel": "Mean silhouette (higher = better separated)"
          },
          "subtitle": "Mean silhouette by number of families, tract data and simulated data with no clusters, with 95% intervals for the sample",
          "title": "The tract data beat simulated data with no clusters by {fact:sil_margin_k} at four families and by at most {fact:sil_margin_other} at any other number",
          "type": "line"
        },
        "label": "Silhouette"
      },
      {
        "figure": {
          "alt": "A line of the median agreement of refits against the number of families, with a band. The line is high up to five families, lower at six, lowest at seven and higher at eight. The band is wide at four families, where its lower end reaches a much lower value than at three or five.",
          "caption": "Median agreement is high up to five families, lower at six and lowest at seven. The band reaches low values at four families because some refits find a different arrangement of the four groups, which is rarer at three and five.",
          "data": [
            {
              "k": 2,
              "median": 0.884321,
              "p10": 0.738151,
              "p90": 0.978066,
              "refits": 200
            },
            {
              "k": 3,
              "median": 0.919263,
              "p10": 0.854682,
              "p90": 0.961051,
              "refits": 200
            },
            {
              "k": 4,
              "median": 0.882047,
              "p10": 0.35186,
              "p90": 0.952537,
              "refits": 200
            },
            {
              "k": 5,
              "median": 0.881051,
              "p10": 0.75314,
              "p90": 0.94738,
              "refits": 200
            },
            {
              "k": 6,
              "median": 0.797961,
              "p10": 0.645019,
              "p90": 0.922063,
              "refits": 200
            },
            {
              "k": 7,
              "median": 0.542914,
              "p10": 0.466688,
              "p90": 0.776997,
              "refits": 200
            },
            {
              "k": 8,
              "median": 0.780306,
              "p10": 0.676799,
              "p90": 0.916538,
              "refits": 200
            }
          ],
          "id": "number_of_families_agreement",
          "spec": {
            "annotations": [
              {
                "label": "Four families",
                "type": "x-rule",
                "x": 4
              }
            ],
            "endLabels": false,
            "hi": "p90",
            "lo": "p10",
            "n": "refits",
            "x": "k",
            "xDomain": [
              2,
              8
            ],
            "xLabel": "Number of families",
            "xTicks": "data",
            "y": "median",
            "yDomain": [
              0,
              1
            ],
            "yFormat": {
              "digits": 1
            },
            "yLabel": "Agreement with the fit on all tracts (adjusted Rand index)"
          },
          "subtitle": "Adjusted Rand index between the fit on all tracts and refits on resamples of counties, by number of families, median and tenth to ninetieth percentile",
          "title": "At four groups {fact:boot_other_share} of refits on resampled counties find a different arrangement, more than at three or five groups",
          "type": "line"
        },
        "label": "Agreement of refits"
      }
    ]
  },
  "subtitle": "Mean silhouette of k-means fits to the {fact:n_measures} standardized deprivation measures, tract data and simulated data with no clusters, and agreement of refits on resamples of counties",
  "title": "The tract data beat a cloud with no clusters by {fact:sil_margin_k} in silhouette at four families, more than at any other number of families",
  "type": "tabs",
  "width": "wide"
}