Classifier#

class glasses_detector.classifier.GlassesClassifier(kind: str = 'anyglasses', size: str = 'medium', weights: bool | str | None = True, device: str | device | None = None)[source]#

Bases: BaseGlassesModel

Binary classifier to check if glasses are present.

This class allows to perform binary classification for images with glasses, i.e., determines whether or not the glasses are present in the image (primarily focus is on whether or not eyeglasses are worn by a person). It is possible to specify a particular kind of glasses to focus on, e.g., sunglasses.

Performance of the Pre-trained Classifiers

Kind	Size	BCE \(\downarrow\)	F1 \(\uparrow\)	ROC-AUC \(\uparrow\)	PR-AUC \(\uparrow\)
`anyglasses`	`small`	0.2160	0.9431	0.9866	0.9757
	`medium`	0.1539	0.9693	0.9933	0.9895
	`large`	TBA	TBA	TBA	TBA
`eyeglasses`	`small`	0.2210	0.9082	0.9808	0.9590
	`medium`	0.1342	0.9502	0.9922	0.9810
	`large`	TBA	0.9490	TBA	TBA
`sunglasses`	`small`	0.2331	0.8827	0.9852	0.9551
	`medium`	0.1794	0.9311	0.9912	0.9739
	`large`	TBA	TBA	TBA	TBA
`shadows`	`small`	0.3956	0.8158	0.9326	0.9075
	`medium`	0.3314	0.8468	0.9537	0.9354
	`large`	TBA	TBA	TBA	TBA

Size Information of the Pre-trained Classifiers

Size	Architecture	Params	GFLOPs	Memory (MB)	Filesize (MB)
`small`	`Tiny Classifier`	0.03M	0.001	23.38	0.12
`medium`	`ShuffleNet` [1]	1.25M	0.19	84.03	4.95
`large`	TBA	TBA	TBA	TBA	TBA

Examples

Let’s instantiate the classifier with default parameters:

>>> from glasses_detector import GlassesClassifier
>>> clf = GlassesClassifier()

First, we can perform a raw prediction on an image expressed as either a path, a PIL Image or a numpy array. See predict() for more details.

>>> clf(np.random.randint(0, 256, size=(224, 224, 3), dtype=np.uint8))
'absent'
>>> clf(["path/to/image1.jpg", "path/to/image2.jpg"], format="bool")
[True, False]

We can also use a more specific method process_file() which allows to save the results to a file:

>>> clf.process_file("path/to/img.jpg", "path/to/pred.txt", show=True)
'present'
>>> clf.process_file(["img1.jpg", "img2.jpg"], "preds.npy", format="proba")
>>> np.load("preds.npy")
array([0.96, 0.81562], dtype=float32)

Finally, we can also use process_dir() to process all images in a directory and save the predictions to a file or a directory:

>>> clf.process_dir("path/to/dir", "path/to/preds.csv", format="str")
>>> subprocess.run(["cat", "path/to/preds.csv"])
img1.jpg,present
img2.jpg,absent
...
>>> clf.process_dir("path/to/dir", "path/to/pred_dir", ext=".txt")
>>> subprocess.run(["ls", "path/to/pred_dir"])
img1.txt img2.txt ...

Parameters:

kind (str, optional) –
The kind of glasses to perform binary classification for. Available options are:

"anyglasses"

Any kind glasses/googles/spectacles

"eyeglasses"

Transparent eyeglasses

"sunglasses"

Opaque and semi-transparent glasses

"shadows"

Visible cast shadows of glasses frames

Each kind is only responsible for its category, e.g., if kind is set to "sunglasses", then images with transparent eyeglasses will not be identified as positive. Defaults to "anyglasses".

size (str, optional) –

The size of the model to use (check ALLOWED_SIZE_ALIASES for size aliases). Available options are:


`"small"` or `"s"`	Very few parameters but lower accuracy
`"medium"` or `"m"`	A balance between the number of parameters and the accuracy
`"large"` or `"l"`	Large number of parameters but higher accuracy

Please check:

Performance of the Pre-trained Classifiers: to see the results of the pre-trained models for each size depending on kind.
Size Information of the Pre-trained Classifiers: to see which architecture each size maps to and the details about the number of parameters.

Defaults to "medium".

weights (bool | str | None, optional) – Whether to load weights from a custom URL (or a local file if they’re already downloaded) which will be inferred based on model’s kind and size. If a string is provided, it will be used as a custom path or a URL (determined automatically) to the model weights. Defaults to True.
device (str | torch.device | None, optional) – Device to cast the model to (once it is loaded). If specified as None, it will be automatically checked if CUDA or MPS is supported. Defaults to None.

static draw_label(image: Image | ndarray | Tensor, label: str, font: str | None = None, font_size: int = 15) → Image[source]#

Draws a label on the image.

This method takes an image and a label and draws a caption box with the given text which is appended to the bottom of the image.

Parameters:

image (PIL.Image.Image | numpy.ndarray | torch.Tensor) – The original image. It can be either a PIL Image, a numpy ndarray of shape (H, W, 3) or (H, W) and type uint8 or a torch Tensor of shape (3, H, W) or (H, W) and type uint8.
label (str) – The label to write in the caption box that is appended to the bottom of the image.
font (str | None, optional) – A filename containing a TrueType font. If the file is not found in this filename, the loader may also search in other directories, such as the fonts/ directory on Windows or /Library/Fonts/, /System/Library/Fonts/ and ~/Library/Fonts/ on macOS. Defaults to None.
font_size (int, optional) – The requested font size in points used when calling truetype(). Defaults to 15.

Returns:

The extended original image in height with the caption box appended to the bottom.

Return type:

PIL.Image.Image

Predicts whether the positive class is present.

Takes a path or multiple paths to image files or the loaded images themselves and outputs a formatted prediction for each image indicating whether it belongs to a positive class, e.g., “anyglasses”, or not. The format of the prediction, i.e., the prediction type is Default type which corresponds to DEFAULT.

Warning

If the image is provided as numpy.ndarray, make sure the last dimension specifies the channels, i.e., last dimension should be of size 1 or 3. If it is anything else, e.g., if the shape is (3, H, W), where W is neither 1 nor 3, this would be interpreted as 3 grayscale images.

Parameters:

image (FilePath | PIL.Image.Image | numpy.ndarray | Collection[FilePath | PIL.Image.Image | numpy.ndarray]) – The path(-s) to the image to generate the prediction for or the image(-s) itself represented as Image or as ndarray. Note that the image should have values between 0 and 255 and be of RGB format. Normalization is not needed as the channels will be automatically normalized before passing through the network.

format (str | dict[bool, Default] | Callable[[torch.Tensor], Default], optional) –

The string specifying the way to map the predictions to labels. These are the following options (if image is a Collection, then the output will be a list of corresponding items of output type):

format	output type	prediction mapping
`"bool"`	`bool`	`True` if positive, `False` if negative
`"int"`	`int`	`1` if positive, `0` if negative
`"str"`	`str`	`"present"` if positive, `"absent"` if negative
`"logit"`	`float`	Raw score (real number) of a positive class
`"proba"`	`float`	Probability (a number between 0 and 1) of a positive class
`"img"`	`PIL.Image.Image`	The original image with an inserted title using default values in `draw_label()` (caption text corresponds to `"str"` format)

It is also possible to provide a dictionary with 2 keys: True and False, each mapping to values corresponding to what to output if the predicted label is positive or negative. Further, a custom callback function is also possible that specifies how to map a raw torch.Tensor score of type torch.float32 of shape (1,) to a label. Defaults to "str".

input_size (tuple[int, int] | None, optional) – The size (width, height), or (W, H), to resize the image to before passing it through the network. If None, the image will not be resized. It is recommended to resize it to the size the model was trained on, which by default is (256, 256). Defaults to (256, 256).

Returns:

The formatted prediction or a list of formatted predictions if multiple images were provided.

Return type:

Default | list[Default]

Raises:

ValueError – If the specified format as a string is not recognized.

classmethod from_model(model: Module, **kwargs) → Self#

Creates a glasses model from a custom torch.nn.Module.

Creates a glasses model wrapper for a custom provided torch.nn.Module, instead of creating a predefined one based on kind and size.

Note

Make sure the provided model’s forward method behaves as expected, i.e., returns the prediction in expected format for compatibility with predict().

Warning

model_info property will not be useful as it would return an empty dictionary for custom specified kind and size (if specified at all).

Parameters:

model (torch.nn.Module) – The custom model that will be assigned as model.
**kwargs – Keyword arguments to pass to the constructor; check the documentation of this class for more details. If task, kind, and size are not provided, they will be set to "custom". If the model architecture is custom, you may still specify the path to the pretrained wights via weights argument. Finally, if device is not provided, the model will remain on the same device as is.

Returns:

The glasses model wrapper of the same class type from which this method was called for the provided custom model.

Return type:

Self

load_weights(path_or_url: str | bool = True)#

Loads inner model weights.

Takes a path of a URL to the weights file, or True to construct the URL automatically based on model_info and loads the weights into model.

Note

If the weights are already downloaded, they will be loaded from the hub cache, which by default is ~/.cache/torch/hub/checkpoints.

Warning

If the fields in model_info are not recognized, e.g., by providing an unrecognized kind or size or by initializing with from_model(), this method will not be able to construct the URL (if path_or_url is True) and will raise a warning.

Parameters:: path_or_url (str | bool, optional) – The path or the URL (it will be inferred automatically) to the model weights (.pth file). It can also be bool, in which case True indicates to construct URL for the pre-trained weights and False does nothing. Defaults to True.

Processes a directory of images.

Takes a path to a directory of images, optionally sub-groups to batches, generates the predictions for every image and returns them if output_path is None or saves them to a specified file or as files to a specified directory. The following cases are considered:

If output_path is None, the predictions are returned as a dictionary of predictions where the keys are the names of the images and the values are the corresponding predictions.
If output_path is a single file, the predictions are aggregated to a single file.
If output_path is a directory, the predictions are saved to that directory. For each input path, a corresponding file is created in the specified output directory with the same name as the input. The extension, if not provided as ext, is set automatically as explained in process_file().

For more details on how each file type is saved, regardless if it is a single prediction or the aggregated predictions, see save().

NB: aggregation of images to a single file/dictionary is different from that of process_file() (when multiple file paths are passed) - here, only the names of the images are used as keys, unlike the full paths.

Tip

For very large directories, consider specifying output_path as a directory because aggregating the predictions to a single file or waiting for them to be returned might consume too much memory and lead to errors.

Note

Any files in the input directory that are not valid images or those for which the prediction fails for any reason are are simply skipped and a warning is raised - for more details, see process_file().

Parameters:

input_path (FilePath) – The path to a directory of images to generate predictions for.
output_path (FilePath | None, optional) – The path to save the prediction(-s) to. If None, the predictions are returned as a dictionary, if a single file, the predictions are aggregated to a single file, and if a directory, the predictions are saved to that directory with the names copied from inputs. Defaults to None.
ext (str | None, optional) – The extension to use for the output file(-s). Only used when output_path is a directory. The extension should include a leading dot, e.g., ".txt", ".npy", ".jpg" etc (see save()). If None, the behavior follows process_file(). Defaults to None.
batch_size (int, optional) – The batch size to use when processing the images. This groups the files in the specified directory to batches of size batch_size before processing them. In some cases, larger batch sizes can speed up the processing at the cost of more memory usage. Defaults to 1.
show (bool, optional) – Whether to show the predictions. Images will be shown using PIL.Image.Image.show() and other predictions will be printed to stdout. It is not recommended to set this to True as it might spam your stdout. Defaults to False.
pbar (bool | str | tqdm, optional) – Whether to show a progress bar. If True, a progress bar with no description is shown. If str, a progress bar with the given description is shown. If an instance of tqdm, it is used as is. Defaults to True.
update_total (bool, optional) – Whether to update the total number of files in the progress bar. This is only relevant if pbar is an instance of tqdm. For example, if the number of total files is already known and captured by tqdm.tqdm.total, then there is no need to update it. Defaults to True.
**pred_kwargs – Additional keyword arguments to pass to predict().

Returns:

The dictionary of predictions if output_path is None or None if output_path is specified.

Return type:

dict[str, Default | None] | None

Processes a single image or a list of images.

Takes a path to the image or a list of paths to images, generates the prediction(-s), and returns them, based on how predict() behaves. If the output path is specified, the prediction(-s) will be saved to the given path(-s) based on the extension of the output path. The following cases are considered:

If output_path is None, no predictions are saved. If there are multiple output paths (one for each input path) and some of the entries are None, then only the outputs for the corresponding predictions are not be saved.
If the output path is a single file, then the predictions are saved to that file. If there are multiple input paths, then the corresponding predictions are aggregated to a single file.
If output_path is a directory, then the prediction(-s) are saved to that directory. For each input path, a corresponding file is created in the specified output directory with the same name as the input. The extension, if not provided as ext, is set to .jpg for images and .txt for other predictions.
If output_path is a list of output paths, then the predictions are saved to the corresponding output paths. If the number of input paths and output paths do not match, then the number of predictions are be truncated or expanded with None to match the number of input paths and a warning is raised. all the output paths are interpreted as files.

For more details on how each file type is saved, regardless if it is a single prediction or the aggregated predictions, see save().

NB: aggregation of multiple images to a single file is different from that of process_dir() - here, the full paths are used as sample identifiers, unlike just the names of the images.

Tip

If multiple images are provided (as a list of input paths), they are likely to be loaded into a single batch for a faster prediction (see predict() for more details), thus more memory is required than if they were processed individually. For this reason, consider not to pass too many images at once (e.g., <200).

Note

If some input path does not lead to a valid image file, e.g., does not exist, its prediction is set to None. Also, if at least one prediction fails, then all predictions are set to None. In both cases, a warning is is raised and the files or the lines in the aggregated file are skipped (not saved).

Parameters:

input_path (FilePath | Collection[FilePath]) – The path to an image or a list of paths to images to generate predictions for.
output_path (FilePath | Collection[FilePath] | None, optional) – The path to save the prediction(-s) to. If None, no predictions are saved. If a single file, the predictions are aggregated (if multiple) and saved to that file. If a directory, the predictions are saved to that directory with the names copied from inputs. Defaults to None.
ext (str | None, optional) – The extension to use for the output file(-s). Only used when output_path is a directory. If None, the extension is set to ".jpg" for images and ".txt" for other predictions (depends on what is returned by predict() returns) For available options, refer to save(). Defaults to None.
show (bool, optional) – Whether to show the predictions. Images will be shown using PIL.Image.Image.show() and other predictions will be printed to stdout. Defaults to False.
**pred_kwargs – Additional keyword arguments to pass to predict().

Returns:

The prediction or a list of predictions for the given image(-s). Any failed predictions will be set to None.

Return type:

Default | None | list[Default | None]


`"anyglasses"`	Any kind glasses/googles/spectacles
`"eyeglasses"`	Transparent eyeglasses
`"sunglasses"`	Opaque and semi-transparent glasses
`"shadows"`	Visible cast shadows of glasses frames