| --- |
| datasets: |
| - CIFAR-10 |
| library_name: tf-keras |
| tags: |
| - clustering |
| --- |
| |
| ## Model description |
|
|
| This is a image clustering model trained after the [**Semantic Clustering by Adopting Nearest neighbors (SCAN)**](https://arxiv.org/abs/2005.12320)(Van Gansbeke et al., 2020) algorithm. |
|
|
| The training procedure was done as seen in the example on <a href='https://keras.io/examples/vision/semantic_image_clustering/' target='_blank'>**keras.io**</a> by [Khalid Salama](https://www.linkedin.com/in/khalid-salama-24403144/). |
|
|
| The algorithm consists of two phases: |
|
|
| 1. Self-supervised visual representation learning of images, in which we use the simCLR technique. |
| 2. Clustering of the learned visual representation vectors to maximize the agreement between the cluster assignments of neighboring vectors. |
|
|
| ## Intended uses & limitations |
|
|
| The model is intended to show the effective use of self-supervised learning combined with nearest neighbours for (semantic) image clustering. |
|
|
| You can use these clusters to retrieve images of the same class. |
|
|
| ### Limitations |
| This model is not supposed to show any superiority to image classification from supervised learning, but as a POC that unsupervised learning is able to cluster similar images together without any labels. |
| ### Possible Improvements: |
| As given by the original author on keras.io, these steps can be taken to improve the accuary further: |
| 1) increase the number of epochs in the representation learning and the clustering phases; |
| 2) allow the encoder weights to be tuned during the clustering phase |
| 3) perform a final fine-tuning step through self-labeling, as described in the original SCAN paper |
|
|
| ## Training and evaluation data |
|
|
| ### Training Data |
| The model was trained using the [CIFAR-10 dataset](https://www.cs.toronto.edu/~kriz/cifar.html). For training the images were scaled to (32,32,3). |
|
|
| ### Hyperparameters |
| For training the following parameters were used: |
| - Feature Vector Dimension: 512 |
| - Projection Units of Head: 128 |
| - Number of Cluster: 20 |
| - K-Neighbours: 5 |
|
|
| The encoder was not tuned during clustering. |
| ### Evaluation |
| #### Visualization of highest confidence cluster picks |
|  |
|
|
| #### Clusters and their respective labels, accuracy and size |
|
|
| | Cluster | Label | Accuracy | Size | |
| |:---------|:-------------:|-----:| -----:| |
| |cluster 0| frog | 31.6 %|3582| |
| |cluster 1| frog | 19.76 %|2348| |
| |cluster 2| horse | 26.82 %|2983| |
| |cluster 3| bird | 29.7 %|1532| |
| |cluster 4| airplane | 39.16 %|3575| |
| |cluster 5| ship | 22.38 %|2207| |
| |cluster 6| automobile | 26.41 %|4365| |
| |cluster 7| dog | 21.09 %|5049| |
| |cluster 8| automobile | 21.94 %|4093| |
| |cluster 9| truck | 29.66 %|4639| |
| |cluster 10| bird | 23.02 %|1455| |
| |cluster 11| truck | 17.78 %|3937| |
| |cluster 12| deer | 30.36 %|2635| |
| |cluster 13| dog | 22.62 %|1950| |
| |cluster 14| frog | 22.64 %|4391| |
| |cluster 15| airplane | 26.89 %|2838| |
| |cluster 16| ship | 34.7 %|2213| |
| |cluster 17| ship | 17.59 %|1785| |
| |cluster 18| cat | 16.57 %|1997| |
| |cluster 19| deer | 27.25 %|2426| |
| ## Model Plot |
|
|
| <details> |
| <summary>View Model Plot</summary> |
|
|
|  |
|
|
| </details> |
