API reference¶
Stimuli¶
corread.stimuli ¶
Word images: single stimuli and reproducible training sets.
A word set is a folder of word images, train/<word>/ and val/<word>/, drawn in given fonts
with random size, case and position. Every random choice is drawn up front from one seed, so a set is
fully determined by its recipe — the seed, the settings, the wordlist and the font files — and can
be drawn again byte for byte (:func:regenerate_training_set). The recipe is written next to the
images:
metadata.json: arguments, seed, rendering settings, words, and the sha256 of every input file;images.csv: the font, size, case and shift drawn for every image;checksums.json: the sha256 digest of every word folder.
generate_image ¶
generate_image(text: str, fontname: str, *, fontsdir: str | None = None, width: int = 500, height: int = 500, size: int = 24, xshift: int = 0, yshift: int = 0, upper: bool = False, mirror: bool = False, invert: bool = False) -> Image.Image
Draw one word image, as the word sets do.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
text
|
str
|
The word. |
required |
fontname
|
str
|
Font file name without |
required |
fontsdir
|
str
|
Folder of the fonts; by default |
None
|
width
|
int
|
Canvas size in pixels. |
500
|
height
|
int
|
Canvas size in pixels. |
500
|
size
|
int
|
Font size. |
24
|
xshift
|
int
|
Offset of the text from the centre of the canvas, in pixels. |
0
|
yshift
|
int
|
Offset of the text from the centre of the canvas, in pixels. |
0
|
upper
|
bool
|
Draw the text uppercased, mirrored left to right, or with its letters reversed. |
False
|
mirror
|
bool
|
Draw the text uppercased, mirrored left to right, or with its letters reversed. |
False
|
invert
|
bool
|
Draw the text uppercased, mirrored left to right, or with its letters reversed. |
False
|
Returns:
| Type | Description |
|---|---|
Image
|
The RGB image. |
generate_training_set ¶
generate_training_set(name: str, wordlist: str, *, train_fonts: list[str], val_fonts: list[str], num_words: int = 1000, num_train: int = 1000, num_val: int = 50, fontsdir: str | None = None, wordsdir: str | None = None, wordsetsdir: str | None = None, width: int = 500, height: int = 500, seed: int | None = None, workers: int = 1, report: bool = False, export: bool = False) -> str
Generate a word set: num_train training and num_val validation images of every word.
The training images use train_fonts and the validation images val_fonts, so validation can
test generalization to an unseen font. Each image draws its font, size (40–80), case, and a shift of
up to ±50 px horizontally and ±30 px vertically.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the set: its folder in |
required |
wordlist
|
str
|
Wordlist file name without |
required |
train_fonts
|
list of str
|
Font file names without |
required |
val_fonts
|
list of str
|
Font file names without |
required |
num_words
|
int
|
Use the first |
1000
|
num_train
|
int
|
Images per word in each split. |
1000
|
num_val
|
int
|
Images per word in each split. |
1000
|
fontsdir
|
str
|
Folders of the fonts, the wordlists and the word sets; by default from |
None
|
wordsdir
|
str
|
Folders of the fonts, the wordlists and the word sets; by default from |
None
|
wordsetsdir
|
str
|
Folders of the fonts, the wordlists and the word sets; by default from |
None
|
width
|
int
|
Canvas size in pixels. |
500
|
height
|
int
|
Canvas size in pixels. |
500
|
seed
|
int
|
Seed of every random choice; a random one is drawn and recorded when not given. |
None
|
workers
|
int
|
Processes drawing images; the result does not depend on it. |
1
|
report
|
bool
|
Send the progress to the reporting API set in |
False
|
export
|
bool
|
Upload the set's recipe to the Hugging Face Hub when it is complete (see :mod: |
False
|
Returns:
| Type | Description |
|---|---|
str
|
The folder of the new set. |
regenerate_training_set ¶
regenerate_training_set(name: str, recipe: str | None = None, *, workers: int = 1, wordsetsdir: str | None = None, fontsdir: str | None = None, wordsdir: str | None = None, allow_mismatch: bool = False) -> str
Draw a word set again from its recipe and check that the images are identical.
The draws are checked against images.csv before any image is drawn, and the images against
checksums.json after. Text rendering and JPEG encoding come from the FreeType and libjpeg that
Pillow bundles, so identical images need the Pillow version recorded in the recipe.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Name of the set; it is drawn into |
required |
recipe
|
str
|
Folder with the recipe files ( |
None
|
workers
|
int
|
Processes drawing images. |
1
|
wordsetsdir
|
str
|
By default from |
None
|
fontsdir
|
str
|
By default from |
None
|
wordsdir
|
str
|
By default from |
None
|
allow_mismatch
|
bool
|
Keep images that differ from the recipe (recorded as |
False
|
Returns:
| Type | Description |
|---|---|
str
|
The folder of the regenerated set. |
report_wordset_status ¶
report_wordset_status(name: str, wordsetsdir: str | None = None) -> None
Send the current status of word set name to the reporting API now; raises if it fails.
Training¶
corread.training.TrainingInstance ¶
Train one model in one mode.
The modes follow Hannagan et al. (2021): a CORnet is first trained on ImageNet (pre), then
either keeps training on images alone (illit) or learns to read: it trains on the images and
the words of a word set together, with the word classes added after the image classes. In
lit_bias the word outputs read only the last 49 dense-layer units; in lit_no_bias they read
all 512.
Each model gets its own folder in the models folder of paths.toml:
pre:pre/<name or pre_z_49>/illit:illiterate/<name or illit_z_79>/lit_bias,lit_no_bias:<name or word set>/lit_bias_z_79/
where z is the model choice and 49 or 79 the last epoch, counted from 0 over the pre-literate
and then the later epochs. The folder holds the finished model <folder name>.pth.tar, a
checkpoint after every epoch, the scores and losses as .npy arrays, and metadata.json:
how the model was trained, on which data (with hashes), by which code, and the history of every
epoch.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model_choice
|
('z', 's')
|
CORnet-Z or CORnet-S. |
'z'
|
mode
|
('pre', 'illit', 'lit_bias', 'lit_no_bias')
|
What to train; the literate and illiterate modes start from a pre-literate model. |
'pre'
|
name
|
str
|
Folder name of a |
None
|
pre_name
|
str
|
The pre-literate model to start from; by default |
None
|
wordset
|
str
|
The word set (literate modes), by its name in the |
None
|
wrd_path
|
str
|
Folders of the word set, the images (ImageNet-style |
None
|
img_path
|
str
|
Folders of the word set, the images (ImageNet-style |
None
|
models_dir
|
str
|
Folders of the word set, the images (ImageNet-style |
None
|
restore_path
|
str
|
A checkpoint to continue from. |
None
|
resume
|
bool
|
Continue an interrupted run from its last checkpoint. |
False
|
export
|
bool
|
Upload the run to the Hugging Face Hub as it trains (see :mod: |
False
|
report
|
bool
|
Send the run's status to the reporting API set in |
False
|
seed
|
int
|
Seed of the run; a random one is drawn and recorded when not given. |
None
|
num_workers
|
int
|
DataLoader worker processes. |
1
|
gpus
|
int or list of int
|
GPUs to train on: all ( |
None
|
batch_size
|
int
|
Training batch size. |
100
|
num_val_items
|
int
|
Validation batch size; with as many val images per class, each batch is one class. |
50
|
max_epochs_pre
|
int
|
Epochs of pre-literate training, and of the literate or illiterate training after it. |
50
|
max_epochs_lit
|
int
|
Epochs of pre-literate training, and of the literate or illiterate training after it. |
50
|
img_classes
|
int
|
Number of image and word classes. |
1000
|
wrd_classes
|
int
|
Number of image and word classes. |
1000
|
lr
|
float
|
SGD settings; there is no learning-rate schedule. |
0.01
|
momentum
|
float
|
SGD settings; there is no learning-rate schedule. |
0.01
|
weight_decay
|
float
|
SGD settings; there is no learning-rate schedule. |
0.01
|
train ¶
train() -> None
Train the model.
Ctrl+C stops cleanly: the DataLoader workers and GPUs are released, and the model folder is
removed if no epoch finished, or kept with status interrupted for resume=True.
is_complete ¶
is_complete() -> bool
Whether this model is already trained; raises if it was trained with different arguments.
report_status ¶
report_status() -> None
Send this model's current status to the reporting API now; raises if it doesn't get through.
Works for any run in this model's folder: finished, interrupted, or training in another process.
A run left in progress by a process that no longer exists on this machine is reported as
stopped.
corread.training.FamilyTrainingInstance ¶
Train a pre-literate model, then the illiterate, unbiased and biased literate models from it.
Takes the arguments of :class:TrainingInstance except mode; pre_name names the
pre-literate model and name the others. A model already trained with these arguments is skipped,
so a family can be run again after an interruption, or for another word set, reusing its
pre-literate and illiterate models.
Attributes:
| Name | Type | Description |
|---|---|---|
instances |
dict
|
The :class: |
train ¶
train() -> None
Train the models in order; stops at the first that fails or is interrupted, since later ones need it.
Lesions¶
corread.lesion.LesionInstance ¶
Bases: TrainingInstance
Lesion dense-layer units of a finished literate CORnet-Z model, then let the model recover.
A lesion re-initialises a unit's incoming weights (its IT conv filter and bias) once, as PyTorch
initialises an untrained layer; afterwards they train like every other weight. The model then keeps
training the way its literate training did (same data, transforms, SGD settings and validation)
for max_epochs epochs.
Before the first epoch the run validates and measures the intact model, lesions it, then validates
and measures the lesioned one; every epoch then trains, validates and measures. To measure is to
record each unit's mean activation to all val words and val images, its mean and SD over the
selectivity sample, and from these its word selectivity (see :meth:word_selectivity), into
unit_tracking.npz.
The run goes into <models>/<group>/<name>/, next to the lesioned model (by default
lesion_<mode>_z_<units>u_<last epoch>), and its epochs continue that model's numbering.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
str
|
The model to lesion, as |
required |
units
|
list of int
|
Dense-layer units to lesion (0..511), e.g. |
required |
name
|
str
|
Folder name of the run. |
None
|
reset_outgoing
|
bool
|
Also re-initialise the units' weights in the readout. |
False
|
max_epochs
|
int
|
Recovery epochs. |
10
|
silent_epochs
|
int
|
Silence the units (multiply their output by 0, so they neither drive the readout nor learn) for
the first |
0
|
threshold
|
float
|
Word selectivity, as for :meth: |
3
|
selectivity_images
|
float
|
Word selectivity, as for :meth: |
3
|
selectivity_seed
|
float
|
Word selectivity, as for :meth: |
3
|
batch_size
|
optional
|
By default the lesioned model's. |
None
|
lr
|
optional
|
By default the lesioned model's. |
None
|
momentum
|
optional
|
By default the lesioned model's. |
None
|
weight_decay
|
optional
|
By default the lesioned model's. |
None
|
wrd_path
|
str
|
The word set; by default the one the lesioned model was trained on. |
None
|
seed
|
int | None
|
As for :class: |
None
|
resume
|
int | None
|
As for :class: |
None
|
report
|
int | None
|
As for :class: |
None
|
models_dir
|
int | None
|
As for :class: |
None
|
img_path
|
int | None
|
As for :class: |
None
|
num_workers
|
int | None
|
As for :class: |
None
|
gpus
|
int | None
|
As for :class: |
None
|
Notes
Hub export is not available for lesion runs.
word_selectivity
staticmethod
¶
word_selectivity(model: str, threshold: float = 3, selectivity_images: int = 5000, selectivity_seed: int = 0, gpus: int | list[int] | None = None, num_workers: int = 1, batch_size: int = 100, models_dir: str | None = None, img_path: str | None = None, wrd_path: str | None = None) -> dict[str, Any]
How word-selective each dense-layer unit of a literate model is.
For unit u: selectivity[u] = (mean activation to the words − mean activation to the images)
/ SD of its activations to the images, where the words are all val word images of the model's word
set and the images the selectivity sample (selectivity_images ImageNet train images drawn with
selectivity_seed), both through the fixed validation view. A unit is word-selective when
selectivity[u] >= threshold.
Returns:
| Type | Description |
|---|---|
dict
|
|
Networks¶
corread.models ¶
The networks: CORnet-Z and CORnet-S, each as an image classifier and as two literate networks.
preandillit: :class:CORnetZ, :class:CORnetSlit_bias: :class:CORnetZBiasedWords, :class:CORnetSBiasedWordslit_no_bias: :class:CORnetZNonbiasedWords, :class:CORnetSNonbiasedWords
Every network returns the activations of all its areas with the outputs; see :meth:CORnet.forward.
NETWORKS
module-attribute
¶
NETWORKS: dict[str, dict[str, type[CORnet]]] = {'z': {'pre': CORnetZ, 'illit': CORnetZ, 'lit_bias': CORnetZBiasedWords, 'lit_no_bias': CORnetZNonbiasedWords}, 's': {'pre': CORnetS, 'illit': CORnetS, 'lit_bias': CORnetSBiasedWords, 'lit_no_bias': CORnetSNonbiasedWords}}
The network class of each model choice ('z', 's') and training mode.
network_class ¶
network_class(model_choice: str, mode: str) -> type[CORnet]
The class of the network that mode trains for CORnet-model_choice.
build_network ¶
build_network(model_choice: str, mode: str, img_classes: int = 1000, wrd_classes: int = 1000, pretrained: ImageClassifier | None = None) -> CORnet
A new network of a mode, optionally started from a pre-literate network (literate modes only).
corread.models.CORnet ¶
Bases: Module
Base class: the ventral stream, the dense layer, and a readout defined by subclasses.
Subclasses set :meth:backbone, which builds new areas, and implement :meth:readout.
backbone
staticmethod
¶
backbone() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]
New, untrained V1, V2, V4 and IT areas.
readout ¶
readout(h: Tensor) -> torch.Tensor
The outputs (logits) from the dense-layer activations h.
forward ¶
forward(image: Tensor) -> Activations
Run a batch of images through the network.
Returns:
| Type | Description |
|---|---|
tuple
|
The flattened activations of V1, V2, V4 and IT, the dense layer |
corread.models.ImageClassifier ¶
Bases: CORnet
The pre-literate (and illiterate) network: every dense-layer unit reads out to the image classes.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
out_img
|
int
|
Number of image classes. |
1000
|
corread.models.BiasedWordClassifier ¶
Bases: CORnet
The biased literate network: the word outputs read only the last 49 dense-layer units.
The image outputs read every unit, as in the pre-literate network; the word outputs read only its
word channel, units 463..511 (:class:~corread.models.layers.BiLinear).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
pretrained
|
ImageClassifier
|
The pre-literate network to start from: its areas are taken over and its image readout copied. |
None
|
out_img
|
int
|
Number of image and word classes; the outputs are the image classes, then the words. |
1000
|
out_wrd
|
int
|
Number of image and word classes; the outputs are the image classes, then the words. |
1000
|
corread.models.UnbiasedWordClassifier ¶
Bases: CORnet
The unbiased literate network: every dense-layer unit reads out to every output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
pretrained
|
ImageClassifier
|
The pre-literate network to start from: its areas are taken over and its readout copied into the image outputs. |
None
|
n_hid
|
int
|
Units of the dense layer. |
DENSE_UNITS
|
out_img
|
int
|
Number of image and word classes; the outputs are the image classes, then the words. |
1000
|
out_wrd
|
int
|
Number of image and word classes; the outputs are the image classes, then the words. |
1000
|
corread.models.cornet_z ¶
CORnet-Z (Kubilius et al., 2018): four feedforward areas of one convolution each.
CORnetZ ¶
Bases: ImageClassifier
CORnet-Z reading out to the image classes only: the pre-literate and illiterate network.
CORnetZBiasedWords ¶
Bases: BiasedWordClassifier
Literate CORnet-Z whose word outputs read only the last 49 dense-layer units.
CORnetZNonbiasedWords ¶
cornet_z_areas ¶
cornet_z_areas() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]
New V1, V2, V4 and IT areas of CORnet-Z.
corread.models.cornet_s ¶
CORnet-S (Kubilius et al., 2019): recurrent, residual areas with batch normalization.
CORnetS ¶
Bases: ImageClassifier
CORnet-S reading out to the image classes only: the pre-literate and illiterate network.
CORnetSBiasedWords ¶
Bases: BiasedWordClassifier
Literate CORnet-S whose word outputs read only the last 49 dense-layer units.
CORnetSNonbiasedWords ¶
cornet_s_areas ¶
cornet_s_areas() -> tuple[nn.Module, nn.Module, nn.Module, nn.Module]
New V1, V2, V4 and IT areas of CORnet-S (V2 and IT unrolled twice, V4 four times).
corread.models.layers ¶
The building blocks of the CORnet networks and their readouts.
DENSE_UNITS
module-attribute
¶
DENSE_UNITS = 512
Units of the dense layer: IT's channels averaged over space, the input of every readout.
WORD_UNITS
module-attribute
¶
WORD_UNITS = 49
Dense-layer units the word outputs of a biased network read: the last 49 (463..511).
Flatten ¶
Bases: Module
Flatten every dimension but the batch.
Identity ¶
Bases: Module
Pass the input through unchanged; a named point to read a block's output from.
BiLinear ¶
Bases: Module
The readout of a biased network.
The image outputs read every dense-layer unit (lin_img); the word outputs read only the last
in_wrd units (lin_wrd), the network's word channel. The outputs are the image outputs
followed by the word outputs.
CORBlockZ ¶
Bases: Module
A CORnet-Z area: convolution, ReLU and max pooling.
CORBlockS ¶
Bases: Module
A recurrent CORnet-S area: a bottleneck residual block applied times times with shared weights.
The first pass downsamples (stride 2) and adds a projected skip connection; the later passes keep the resolution and add the identity. Each pass has its own batch normalization.
cornet_s_v1 ¶
cornet_s_v1() -> nn.Sequential
V1 of CORnet-S: two convolutions with a max pool between them.
Sharing through the Hugging Face Hub¶
corread.hub ¶
Share trained models and word sets through a Hugging Face organisation of your own.
Set the organisation (or your user name) in paths.toml, and sign in once with hf auth login
(or set HF_TOKEN; the token never goes in paths.toml)::
[hub]
org = "my-lab"
private = true # visibility of the repos created from here on
The organisation holds three kinds of repo:
<org>/<group>.<name>(model): the model inmodels/<group>/<name>: itsmetadata.jsonfrom the first epoch on, and once complete the model file, scores and log; never the checkpoints.<org>/<word set>(dataset): the word set's recipe, never its images; :func:import_wordsetdraws them again, byte for byte.<org>/assets(dataset, always private): the fonts and wordlists the recipes are drawn from.
Every repo gets a generated card; a model's card links the pre-literate model it started from and the
word set it was trained on, so the Hub shows the lineage of every model. See docs/hub.md.
export_model ¶
export_model(model: str, overwrite: bool = False) -> str
Upload the model folder models/<model> ('<group>/<name>', e.g. 'pre/pre_z_49').
Uploads metadata.json, and once training is complete also the model file, scores and log. A
repo holding another run is an error unless overwrite. TrainingInstance(export=True) does
this as it trains. Returns the repo id.
import_model ¶
import_model(model: str) -> str
Download the finished model <org>/<group>.<name> into models/<model>; returns the folder.
The model file is checked against the hash in its metadata, and the folder only appears once everything is in place.
export_wordset ¶
export_wordset(name: str, overwrite: bool = False) -> str
Upload the recipe of the word set wordsets/<name>, and its fonts and wordlist to <org>/assets.
The recipe pins the revision of <org>/assets that holds its inputs, so a font replaced there
later does not change what the recipe regenerates. Returns the repo id.
import_wordset ¶
import_wordset(name: str, workers: int = 1, allow_mismatch: bool = False) -> str
Download the recipe of word set name and draw its images again into wordsets/<name>.
Every word folder is checked against the recipe's checksums; see
:func:corread.stimuli.regenerate_training_set. Returns the folder.
export_assets ¶
export_assets(overwrite: bool = False) -> str
Upload every font (fonts/*.ttf) and wordlist (wordlists/*.txt) to the private <org>/assets.
A file already there with different content is an error unless overwrite (word set recipes keep
pointing to the old one). Returns the revision that holds them all.
import_assets ¶
import_assets(overwrite: bool = False) -> list[str]
Download every font and wordlist of <org>/assets into your fonts and wordlists folders.
A local file with different content is an error unless overwrite. Returns the files copied.
list_models ¶
list_models() -> list[dict[str, Any]]
The models in the organisation: model ('<group>/<name>'), repo, url, status, mode, accuracies, ...
list_wordsets ¶
list_wordsets() -> list[dict[str, Any]]
The word set recipes in the organisation: name, repo, url, wordlist, fonts, number of images, ...
ImageNet¶
corread.imagenet ¶
Turn the ImageNet (ILSVRC 2012) archives into the train/<wnid>/ and val/<wnid>/ folders training reads.
ImageNet is not distributed with CORread: download ILSVRC2012_img_train.tar,
ILSVRC2012_img_val.tar and ILSVRC2012_devkit_t12.tar.gz from https://image-net.org under its
terms of access, then run :func:prepare_imagenet.
prepare_imagenet ¶
prepare_imagenet(folder: str | None = None, keep_archives: bool = False, workers: int = 16) -> None
Unpack the ImageNet archives in folder (by default imagesets from paths.toml).
The archives are used up as they are unpacked (keep_archives=True keeps them, but then the disk
needs room for both). Entries are written only as plain files inside their class folders, never
where an archive's paths point. Safe to rerun after an interruption: it continues where it stopped.
Configuration¶
corread.paths ¶
Where CORread finds its inputs and writes its outputs: the paths.toml configuration file.
The file is the one named by the CORREAD_PATHS environment variable, or else the first
paths.toml in the working directory or one of its parents. Its top-level entries are folders,
relative to the file; its tables ([hub], [report]) hold the settings of optional features.
See examples/paths.example.toml.
get_path ¶
get_path(name: str) -> str
The absolute path of the folder name (fonts, wordlists, imagesets, wordsets, models).
get_section ¶
get_section(name: str) -> dict[str, Any]
The table [name] of the configuration, e.g. the [hub] settings.
clear_cache ¶
clear_cache() -> None
Forget the configuration read so far, e.g. after changing CORREAD_PATHS.