Skip to content

API reference

Stimuli

corread.stimuli

Word images: single stimuli and reproducible training sets.

A word set is a folder of word images, train/<word>/ and val/<word>/, drawn in given fonts with random size, case and position. Every random choice is drawn up front from one seed, so a set is fully determined by its recipe — the seed, the settings, the wordlist and the font files — and can be drawn again byte for byte (:func:regenerate_training_set). The recipe is written next to the images:

  • metadata.json: arguments, seed, rendering settings, words, and the sha256 of every input file;
  • images.csv: the font, size, case and shift drawn for every image;
  • checksums.json: the sha256 digest of every word folder.

generate_image

generate_image(text: str, fontname: str, *, fontsdir: str | None = None, width: int = 500, height: int = 500, size: int = 24, xshift: int = 0, yshift: int = 0, upper: bool = False, mirror: bool = False, invert: bool = False) -> Image.Image

Draw one word image, as the word sets do.

Parameters:

Name Type Description Default
text str

The word.

required
fontname str

Font file name without .ttf, in fontsdir.

required
fontsdir str

Folder of the fonts; by default fonts from paths.toml.

None
width int

Canvas size in pixels.

500
height int

Canvas size in pixels.

500
size int

Font size.

24
xshift int

Offset of the text from the centre of the canvas, in pixels.

0
yshift int

Offset of the text from the centre of the canvas, in pixels.

0
upper bool

Draw the text uppercased, mirrored left to right, or with its letters reversed.

False
mirror bool

Draw the text uppercased, mirrored left to right, or with its letters reversed.

False
invert bool

Draw the text uppercased, mirrored left to right, or with its letters reversed.

False

Returns:

Type Description
Image

The RGB image.

generate_training_set

generate_training_set(name: str, wordlist: str, *, train_fonts: list[str], val_fonts: list[str], num_words: int = 1000, num_train: int = 1000, num_val: int = 50, fontsdir: str | None = None, wordsdir: str | None = None, wordsetsdir: str | None = None, width: int = 500, height: int = 500, seed: int | None = None, workers: int = 1, report: bool = False, export: bool = False) -> str

Generate a word set: num_train training and num_val validation images of every word.

The training images use train_fonts and the validation images val_fonts, so validation can test generalization to an unseen font. Each image draws its font, size (40–80), case, and a shift of up to ±50 px horizontally and ±30 px vertically.

Parameters:

Name Type Description Default
name str

Name of the set: its folder in wordsetsdir.

required
wordlist str

Wordlist file name without .txt, in wordsdir: one word per line.

required
train_fonts list of str

Font file names without .ttf, in fontsdir.

required
val_fonts list of str

Font file names without .ttf, in fontsdir.

required
num_words int

Use the first num_words words of the wordlist.

1000
num_train int

Images per word in each split.

1000
num_val int

Images per word in each split.

1000
fontsdir str

Folders of the fonts, the wordlists and the word sets; by default from paths.toml.

None
wordsdir str

Folders of the fonts, the wordlists and the word sets; by default from paths.toml.

None
wordsetsdir str

Folders of the fonts, the wordlists and the word sets; by default from paths.toml.

None
width int

Canvas size in pixels.

500
height int

Canvas size in pixels.

500
seed int

Seed of every random choice; a random one is drawn and recorded when not given.

None
workers int

Processes drawing images; the result does not depend on it.

1
report bool

Send the progress to the reporting API set in paths.toml (see docs/reporting.md).

False
export bool

Upload the set's recipe to the Hugging Face Hub when it is complete (see :mod:corread.hub).

False

Returns:

Type Description
str

The folder of the new set.

regenerate_training_set

regenerate_training_set(name: str, recipe: str | None = None, *, workers: int = 1, wordsetsdir: str | None = None, fontsdir: str | None = None, wordsdir: str | None = None, allow_mismatch: bool = False) -> str

Draw a word set again from its recipe and check that the images are identical.

The draws are checked against images.csv before any image is drawn, and the images against checksums.json after. Text rendering and JPEG encoding come from the FreeType and libjpeg that Pillow bundles, so identical images need the Pillow version recorded in the recipe.

Parameters:

Name Type Description Default
name str

Name of the set; it is drawn into wordsetsdir/name, which must not exist.

required
recipe str

Folder with the recipe files (metadata.json, images.csv, checksums.json); an existing set's folder works too. By default wordsetsdir/<name>.recipe, where :func:corread.hub.import_wordset puts them.

None
workers int

Processes drawing images.

1
wordsetsdir str

By default from paths.toml. The fonts and wordlist must be the files the set was made with.

None
fontsdir str

By default from paths.toml. The fonts and wordlist must be the files the set was made with.

None
wordsdir str

By default from paths.toml. The fonts and wordlist must be the files the set was made with.

None
allow_mismatch bool

Keep images that differ from the recipe (recorded as verified: false in regeneration.json) instead of raising.

False

Returns:

Type Description
str

The folder of the regenerated set.

report_wordset_status

report_wordset_status(name: str, wordsetsdir: str | None = None) -> None

Send the current status of word set name to the reporting API now; raises if it fails.

Training

corread.training.TrainingInstance

Train one model in one mode.

The modes follow Hannagan et al. (2021): a CORnet is first trained on ImageNet (pre), then either keeps training on images alone (illit) or learns to read: it trains on the images and the words of a word set together, with the word classes added after the image classes. In lit_bias the word outputs read only the last 49 dense-layer units; in lit_no_bias they read all 512.

Each model gets its own folder in the models folder of paths.toml:

  • pre: pre/<name or pre_z_49>/
  • illit: illiterate/<name or illit_z_79>/
  • lit_bias, lit_no_bias: <name or word set>/lit_bias_z_79/

where z is the model choice and 49 or 79 the last epoch, counted from 0 over the pre-literate and then the later epochs. The folder holds the finished model <folder name>.pth.tar, a checkpoint after every epoch, the scores and losses as .npy arrays, and metadata.json: how the model was trained, on which data (with hashes), by which code, and the history of every epoch.

Parameters:

Name Type Description Default
model_choice ('z', 's')

CORnet-Z or CORnet-S.

'z'
mode ('pre', 'illit', 'lit_bias', 'lit_no_bias')

What to train; the literate and illiterate modes start from a pre-literate model.

'pre'
name str

Folder name of a pre or illit model; for the literate modes the name of their group folder (by default the word set's name).

None
pre_name str

The pre-literate model to start from; by default pre_<model_choice>_<max_epochs_pre - 1>.

None
wordset str

The word set (literate modes), by its name in the wordsets folder.

None
wrd_path str

Folders of the word set, the images (ImageNet-style train/ and val/) and the models; by default from paths.toml.

None
img_path str

Folders of the word set, the images (ImageNet-style train/ and val/) and the models; by default from paths.toml.

None
models_dir str

Folders of the word set, the images (ImageNet-style train/ and val/) and the models; by default from paths.toml.

None
restore_path str

A checkpoint to continue from.

None
resume bool

Continue an interrupted run from its last checkpoint.

False
export bool

Upload the run to the Hugging Face Hub as it trains (see :mod:corread.hub).

False
report bool

Send the run's status to the reporting API set in paths.toml (see :mod:corread.reporting).

False
seed int

Seed of the run; a random one is drawn and recorded when not given.

None
num_workers int

DataLoader worker processes.

1
gpus int or list of int

GPUs to train on: all (None), the first n, or these indices; 0 trains on the CPU.

None
batch_size int

Training batch size.

100
num_val_items int

Validation batch size; with as many val images per class, each batch is one class.

50
max_epochs_pre int

Epochs of pre-literate training, and of the literate or illiterate training after it.

50
max_epochs_lit int

Epochs of pre-literate training, and of the literate or illiterate training after it.

50
img_classes int

Number of image and word classes.

1000
wrd_classes int

Number of image and word classes.

1000
lr float

SGD settings; there is no learning-rate schedule.

0.01
momentum float

SGD settings; there is no learning-rate schedule.

0.01
weight_decay float

SGD settings; there is no learning-rate schedule.

0.01

train

train() -> None

Train the model.

Ctrl+C stops cleanly: the DataLoader workers and GPUs are released, and the model folder is removed if no epoch finished, or kept with status interrupted for resume=True.

is_complete

is_complete() -> bool

Whether this model is already trained; raises if it was trained with different arguments.

report_status

report_status() -> None

Send this model's current status to the reporting API now; raises if it doesn't get through.

Works for any run in this model's folder: finished, interrupted, or training in another process. A run left in progress by a process that no longer exists on this machine is reported as stopped.

corread.training.FamilyTrainingInstance

Train a pre-literate model, then the illiterate, unbiased and biased literate models from it.

Takes the arguments of :class:TrainingInstance except mode; pre_name names the pre-literate model and name the others. A model already trained with these arguments is skipped, so a family can be run again after an interruption, or for another word set, reusing its pre-literate and illiterate models.

Attributes:

Name Type Description
instances dict

The :class:TrainingInstance of each mode, in training order.

train

train() -> None

Train the models in order; stops at the first that fails or is interrupted, since later ones need it.

Lesions

corread.lesion.LesionInstance

Bases: TrainingInstance

Lesion dense-layer units of a finished literate CORnet-Z model, then let the model recover.

A lesion re-initialises a unit's incoming weights (its IT conv filter and bias) once, as PyTorch initialises an untrained layer; afterwards they train like every other weight. The model then keeps training the way its literate training did (same data, transforms, SGD settings and validation) for max_epochs epochs.

Before the first epoch the run validates and measures the intact model, lesions it, then validates and measures the lesioned one; every epoch then trains, validates and measures. To measure is to record each unit's mean activation to all val words and val images, its mean and SD over the selectivity sample, and from these its word selectivity (see :meth:word_selectivity), into unit_tracking.npz.

The run goes into <models>/<group>/<name>/, next to the lesioned model (by default lesion_<mode>_z_<units>u_<last epoch>), and its epochs continue that model's numbering.

Parameters:

Name Type Description Default
model str

The model to lesion, as '<group>/<name>' in the models folder, e.g. 'english/lit_bias_z_79'.

required
units list of int

Dense-layer units to lesion (0..511), e.g. word_selectivity(model)['selective_units']. [] trains the same way without a lesion, as a control.

required
name str

Folder name of the run.

None
reset_outgoing bool

Also re-initialise the units' weights in the readout.

False
max_epochs int

Recovery epochs.

10
silent_epochs int

Silence the units (multiply their output by 0, so they neither drive the readout nor learn) for the first silent_epochs recovery epochs, and re-initialise them when they are released; max_epochs makes the lesion permanent.

0
threshold float

Word selectivity, as for :meth:word_selectivity.

3
selectivity_images float

Word selectivity, as for :meth:word_selectivity.

3
selectivity_seed float

Word selectivity, as for :meth:word_selectivity.

3
batch_size optional

By default the lesioned model's.

None
lr optional

By default the lesioned model's.

None
momentum optional

By default the lesioned model's.

None
weight_decay optional

By default the lesioned model's.

None
wrd_path str

The word set; by default the one the lesioned model was trained on.

None
seed int | None

As for :class:~corread.training.TrainingInstance.

None
resume int | None

As for :class:~corread.training.TrainingInstance.

None
report int | None

As for :class:~corread.training.TrainingInstance.

None
models_dir int | None

As for :class:~corread.training.TrainingInstance.

None
img_path int | None

As for :class:~corread.training.TrainingInstance.

None
num_workers int | None

As for :class:~corread.training.TrainingInstance.

None
gpus int | None

As for :class:~corread.training.TrainingInstance.

None
Notes

Hub export is not available for lesion runs.

word_selectivity staticmethod

word_selectivity(model: str, threshold: float = 3, selectivity_images: int = 5000, selectivity_seed: int = 0, gpus: int | list[int] | None = None, num_workers: int = 1, batch_size: int = 100, models_dir: str | None = None, img_path: str | None = None, wrd_path: str | None = None) -> dict[str, Any]

How word-selective each dense-layer unit of a literate model is.

For unit u: selectivity[u] = (mean activation to the words − mean activation to the images) / SD of its activations to the images, where the words are all val word images of the model's word set and the images the selectivity sample (selectivity_images ImageNet train images drawn with selectivity_seed), both through the fixed validation view. A unit is word-selective when selectivity[u] >= threshold.

Returns:

Type Description
dict

mean_words, mean_images, sd_images, selectivity, word_selective: arrays over the 512 units; selective_units: the word-selective units, most selective first; ranking: all units, most selective first; word_channel: the units the biased model's word outputs read, else None; and model, threshold, num_words, num_images, selectivity_seed.

Networks

corread.models

The networks: CORnet-Z and CORnet-S, each as an image classifier and as two literate networks.

  • pre and illit: :class:CORnetZ, :class:CORnetS
  • lit_bias: :class:CORnetZBiasedWords, :class:CORnetSBiasedWords
  • lit_no_bias: :class:CORnetZNonbiasedWords, :class:CORnetSNonbiasedWords

Every network returns the activations of all its areas with the outputs; see :meth:CORnet.forward.

NETWORKS module-attribute

NETWORKS: dict[str, dict[str, type[CORnet]]] = {'z': {'pre': CORnetZ, 'illit': CORnetZ, 'lit_bias': CORnetZBiasedWords, 'lit_no_bias': CORnetZNonbiasedWords}, 's': {'pre': CORnetS, 'illit': CORnetS, 'lit_bias': CORnetSBiasedWords, 'lit_no_bias': CORnetSNonbiasedWords}}

The network class of each model choice ('z', 's') and training mode.

network_class

network_class(model_choice: str, mode: str) -> type[CORnet]

The class of the network that mode trains for CORnet-model_choice.

build_network

build_network(model_choice: str, mode: str, img_classes: int = 1000, wrd_classes: int = 1000, pretrained: ImageClassifier | None = None) -> CORnet

A new network of a mode, optionally started from a pre-literate network (literate modes only).

corread.models.CORnet

Bases: Module

Base class: the ventral stream, the dense layer, and a readout defined by subclasses.

Subclasses set :meth:backbone, which builds new areas, and implement :meth:readout.

backbone staticmethod

backbone() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]

New, untrained V1, V2, V4 and IT areas.

readout

readout(h: Tensor) -> torch.Tensor

The outputs (logits) from the dense-layer activations h.

forward

forward(image: Tensor) -> Activations

Run a batch of images through the network.

Returns:

Type Description
tuple

The flattened activations of V1, V2, V4 and IT, the dense layer h (batch × 512) and the outputs (batch × number of classes).

corread.models.ImageClassifier

Bases: CORnet

The pre-literate (and illiterate) network: every dense-layer unit reads out to the image classes.

Parameters:

Name Type Description Default
out_img int

Number of image classes.

1000

corread.models.BiasedWordClassifier

Bases: CORnet

The biased literate network: the word outputs read only the last 49 dense-layer units.

The image outputs read every unit, as in the pre-literate network; the word outputs read only its word channel, units 463..511 (:class:~corread.models.layers.BiLinear).

Parameters:

Name Type Description Default
pretrained ImageClassifier

The pre-literate network to start from: its areas are taken over and its image readout copied.

None
out_img int

Number of image and word classes; the outputs are the image classes, then the words.

1000
out_wrd int

Number of image and word classes; the outputs are the image classes, then the words.

1000

corread.models.UnbiasedWordClassifier

Bases: CORnet

The unbiased literate network: every dense-layer unit reads out to every output.

Parameters:

Name Type Description Default
pretrained ImageClassifier

The pre-literate network to start from: its areas are taken over and its readout copied into the image outputs.

None
n_hid int

Units of the dense layer.

DENSE_UNITS
out_img int

Number of image and word classes; the outputs are the image classes, then the words.

1000
out_wrd int

Number of image and word classes; the outputs are the image classes, then the words.

1000

corread.models.cornet_z

CORnet-Z (Kubilius et al., 2018): four feedforward areas of one convolution each.

CORnetZ

Bases: ImageClassifier

CORnet-Z reading out to the image classes only: the pre-literate and illiterate network.

CORnetZBiasedWords

Bases: BiasedWordClassifier

Literate CORnet-Z whose word outputs read only the last 49 dense-layer units.

CORnetZNonbiasedWords

Bases: UnbiasedWordClassifier

Literate CORnet-Z whose word outputs read every dense-layer unit.

cornet_z_areas

cornet_z_areas() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]

New V1, V2, V4 and IT areas of CORnet-Z.

corread.models.cornet_s

CORnet-S (Kubilius et al., 2019): recurrent, residual areas with batch normalization.

CORnetS

Bases: ImageClassifier

CORnet-S reading out to the image classes only: the pre-literate and illiterate network.

CORnetSBiasedWords

Bases: BiasedWordClassifier

Literate CORnet-S whose word outputs read only the last 49 dense-layer units.

CORnetSNonbiasedWords

Bases: UnbiasedWordClassifier

Literate CORnet-S whose word outputs read every dense-layer unit.

cornet_s_areas

cornet_s_areas() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]

New V1, V2, V4 and IT areas of CORnet-S (V2 and IT unrolled twice, V4 four times).

corread.models.layers

The building blocks of the CORnet networks and their readouts.

DENSE_UNITS module-attribute

DENSE_UNITS = 512

Units of the dense layer: IT's channels averaged over space, the input of every readout.

WORD_UNITS module-attribute

WORD_UNITS = 49

Dense-layer units the word outputs of a biased network read: the last 49 (463..511).

Flatten

Bases: Module

Flatten every dimension but the batch.

Identity

Bases: Module

Pass the input through unchanged; a named point to read a block's output from.

BiLinear

Bases: Module

The readout of a biased network.

The image outputs read every dense-layer unit (lin_img); the word outputs read only the last in_wrd units (lin_wrd), the network's word channel. The outputs are the image outputs followed by the word outputs.

CORBlockZ

Bases: Module

A CORnet-Z area: convolution, ReLU and max pooling.

CORBlockS

Bases: Module

A recurrent CORnet-S area: a bottleneck residual block applied times times with shared weights.

The first pass downsamples (stride 2) and adds a projected skip connection; the later passes keep the resolution and add the identity. Each pass has its own batch normalization.

cornet_s_v1

cornet_s_v1() -> nn.Sequential

V1 of CORnet-S: two convolutions with a max pool between them.

Sharing through the Hugging Face Hub

corread.hub

Share trained models and word sets through a Hugging Face organisation of your own.

Set the organisation (or your user name) in paths.toml, and sign in once with hf auth login (or set HF_TOKEN; the token never goes in paths.toml)::

[hub]
org = "my-lab"
private = true    # visibility of the repos created from here on

The organisation holds three kinds of repo:

  • <org>/<group>.<name> (model): the model in models/<group>/<name>: its metadata.json from the first epoch on, and once complete the model file, scores and log; never the checkpoints.
  • <org>/<word set> (dataset): the word set's recipe, never its images; :func:import_wordset draws them again, byte for byte.
  • <org>/assets (dataset, always private): the fonts and wordlists the recipes are drawn from.

Every repo gets a generated card; a model's card links the pre-literate model it started from and the word set it was trained on, so the Hub shows the lineage of every model. See docs/hub.md.

export_model

export_model(model: str, overwrite: bool = False) -> str

Upload the model folder models/<model> ('<group>/<name>', e.g. 'pre/pre_z_49').

Uploads metadata.json, and once training is complete also the model file, scores and log. A repo holding another run is an error unless overwrite. TrainingInstance(export=True) does this as it trains. Returns the repo id.

import_model

import_model(model: str) -> str

Download the finished model <org>/<group>.<name> into models/<model>; returns the folder.

The model file is checked against the hash in its metadata, and the folder only appears once everything is in place.

export_wordset

export_wordset(name: str, overwrite: bool = False) -> str

Upload the recipe of the word set wordsets/<name>, and its fonts and wordlist to <org>/assets.

The recipe pins the revision of <org>/assets that holds its inputs, so a font replaced there later does not change what the recipe regenerates. Returns the repo id.

import_wordset

import_wordset(name: str, workers: int = 1, allow_mismatch: bool = False) -> str

Download the recipe of word set name and draw its images again into wordsets/<name>.

Every word folder is checked against the recipe's checksums; see :func:corread.stimuli.regenerate_training_set. Returns the folder.

export_assets

export_assets(overwrite: bool = False) -> str

Upload every font (fonts/*.ttf) and wordlist (wordlists/*.txt) to the private <org>/assets.

A file already there with different content is an error unless overwrite (word set recipes keep pointing to the old one). Returns the revision that holds them all.

import_assets

import_assets(overwrite: bool = False) -> list[str]

Download every font and wordlist of <org>/assets into your fonts and wordlists folders.

A local file with different content is an error unless overwrite. Returns the files copied.

list_models

list_models() -> list[dict[str, Any]]

The models in the organisation: model ('<group>/<name>'), repo, url, status, mode, accuracies, ...

list_wordsets

list_wordsets() -> list[dict[str, Any]]

The word set recipes in the organisation: name, repo, url, wordlist, fonts, number of images, ...

ImageNet

corread.imagenet

Turn the ImageNet (ILSVRC 2012) archives into the train/<wnid>/ and val/<wnid>/ folders training reads.

ImageNet is not distributed with CORread: download ILSVRC2012_img_train.tar, ILSVRC2012_img_val.tar and ILSVRC2012_devkit_t12.tar.gz from https://image-net.org under its terms of access, then run :func:prepare_imagenet.

prepare_imagenet

prepare_imagenet(folder: str | None = None, keep_archives: bool = False, workers: int = 16) -> None

Unpack the ImageNet archives in folder (by default imagesets from paths.toml).

The archives are used up as they are unpacked (keep_archives=True keeps them, but then the disk needs room for both). Entries are written only as plain files inside their class folders, never where an archive's paths point. Safe to rerun after an interruption: it continues where it stopped.

Configuration

corread.paths

Where CORread finds its inputs and writes its outputs: the paths.toml configuration file.

The file is the one named by the CORREAD_PATHS environment variable, or else the first paths.toml in the working directory or one of its parents. Its top-level entries are folders, relative to the file; its tables ([hub], [report]) hold the settings of optional features. See examples/paths.example.toml.

get_path

get_path(name: str) -> str

The absolute path of the folder name (fonts, wordlists, imagesets, wordsets, models).

get_section

get_section(name: str) -> dict[str, Any]

The table [name] of the configuration, e.g. the [hub] settings.

find_config

find_config() -> str

The absolute path of the configuration file in use.

clear_cache

clear_cache() -> None

Forget the configuration read so far, e.g. after changing CORREAD_PATHS.