scikit-learn Summary of model persistence methods widths 25 50 50 header rows 1 modelpersistence Model persistence

Answer:
.. _model_persistence:

=================
Model persistence
=================

.. list-table:: Summary of model persistence methods
   :widths: 25 50 50
   :header-rows: 1

   * - Persistence method
     - Pros
     - Risks / Cons
   * - :ref:`ONNX <onnx_persistence>`
     - * Serve models without a Python environment
       * Serving and training environments independent of one another
       * Most secure option
     - * Not all scikit-learn models are supported
       * Custom estimators require more work to support
       * Original Python object is lost and cannot be reconstructed
   * - :ref:`skops_persistence`
     - * More secure than `pickle` based formats
       * Contents can be partly validated without loading
     - * Not as fast as `pickle` based formats
       * Supports less types than `pickle` based formats
       * Requires the same environment as the training environment
   * - :mod:`pickle`
     - * Native to Python
       * Can serialize most Python objects
       * Efficient memory usage with `protocol=5`
     - * Loading can execute arbitrary code
       * Requires the same environment as the training environment
   * - :mod:`joblib`
     - * Efficient memory usage
       * Supports memory mapping
       * Easy shortcuts for compression and decompression
     - * Pickle based format
       * Loading can execute arbitrary code
       * Requires the same environment as the training environment
   * - `cloudpickle`_
     - * Can serialize non-packaged, custom Python code
       * Comparable loading efficiency as :mod:`pickle` with `protocol=5`
     - * Pickle based format
       * Loading can execute arbitrary code
       * No forward compatibility guarantees
       * Requires the same environment as the training environment

After training a scikit-learn model, it is desirable to have a way to persist
the model for future use without having to retrain. Based on your use-case,
there are a few different ways to persist a scikit-learn model, and here we
help you decide which one suits you best. In order to make a decision, you need
to answer the following questions:

1. Do you need the Python object after persistence, or do you only need to
   persist in order to serve the model and get predictions out of it?

If you only need to serve the model and no further investigation on the Python
object itself is required, then :ref:`ONNX <onnx_persistence>` might be the
best fit for you. Note that not all models are supported by ONNX.

In case ONNX is not suitable for your use-case, the next question is:

2. Do you absolutely trust the source of the model, or are there any security
   concerns regarding where the persisted model comes from?

If you have security concerns, then you should consider using :ref:`skops.io
<skops_persistence>` which gives you back the Python object, but unlike
`pickle` based persistence solutions, loading the persisted model doesn't
automatically allow arbitrary code execution. Note that this requires manual
investigation of the persisted file, which :mod:`skops.io` allows you to do.

The other solutions assume you absolutely trust the source of the file to be
loaded, as they are all susceptible to arbitrary code execution upon loading
the persisted file since they all use the pickle protocol under the hood.

3. Do you care about the performance of loading the model, and sharing it
   between processes where a memory mapped object on disk is beneficial?

If yes, then you can consider using :ref:`joblib <pickle_persistence>`. If this
is not a major concern for you, then you can use the built-in :mod:`pickle`
module.

4. Did you try :mod:`pickle` or :mod:`joblib` and found that the model cannot
   be persisted? It can happen for instance when you have user defined
   functions in your model.

If yes, then you can use `cloudpickle`_ which can serialize certain objects
which cannot be serialized by :mod:`pickle` or :mod:`joblib`.

Workflow Overview
-----------------

In a typical workflow, the first step is to train the model using scikit-learn
and scikit-learn compatible libraries. Note that support for scikit-learn and
third party estimators varies across the different persistence methods.

Train and Persist the Model
...........................

Creating an appropriate model depends on your use-case. As an example, here we
train a :class:`sklearn.ensemble.HistGradientBoostingClassifier` on the iris
dataset::

  >>> from sklearn import ensemble
  >>> from sklearn import datasets
  >>> clf = ensemble.HistGradientBoostingClassifier()
  >>> X, y = datasets.load_iris(return_X_y=True)
  >>> clf.fit(X, y)
  HistGradientBoostingClassifier()

Once the model is trained, you can persist it using your desired method, and
then you can load the model in a separate environment and get predictions from
it given input data. Here there are two major paths depending on how you
persist and plan to serve the model:

- :ref:`ONNX <onnx_persistence>`: You need an `ONNX` runtime and an environment
  with appropriate dependencies installed to load the model and use the runtime
  to get predictions. This environment can be minimal and does not necessarily
  even require Python to be installed to load the model and compute
  predictions. Also note that `onnxruntime` typically requires much less RAM
  than Python to compute predictions from small models.

- :mod:`skops.io`, :mod:`pickle`, :mod:`joblib`, `cloudpickle`_: You need a
  Python environment with the appropriate dependencies installed to load the
  model and get predictions from it. This environment should have the same
  **packages** and the same **versions** as the environment where the model was
  trained. Note that none of these methods support loading a model trained with
  a different version of scikit-learn, and possibly different versions of other
  dependencies such as `numpy` and `scipy`. Another concern would be running
  the persisted model on a different hardware, and in most cases you should be
  able to load your persisted model on a different hardware.

.. _onnx_persistence:

ONNX
----

`ONNX`, or `Open Neural Network Exchange <https://onnx.ai/>`__ format is best
suitable in use-cases where one needs to persist the model and then use the
persisted artifact to get predictions without the need to load the Python
object itself. It is also useful in cases where the serving environment needs
to be lean and minimal, since the `ONNX` runtime does not require `python`.

`ONNX` is a binary serialization of the model. It has been developed to improve
the usability of the interoperable representation of data models. It aims to
facilitate the conversion of the data models between different machine learning
frameworks, and to improve their portability on different computing
architectures. More details are available from the `ONNX tutorial
<https://onnx.ai/get-started.html>`__. To convert scikit-learn model to `ONNX`
`sklearn-onnx <http://onnx.ai/sklearn-onnx/>`__ has been developed. However,
not all scikit-learn models are supported, and it is limited to the core
scikit-learn and does not support most third party estimators. One can write a
custom converter for third party or custom estimators, but the documentation to
do that is sparse and it might be challenging to do so.

.. dropdown:: Using ONNX

  To convert the model to `ONNX` format, you need to give the converter some
  information about the input as well, about which you can read more `here
  <http://onnx.ai/sklearn-onnx/index.html>`__::

      from skl2onnx import to_onnx
      onx = to_onnx(clf, X[:1].astype(numpy.float32), target_opset=12)
      with open("filename.onnx", "wb") as f:
          f.write(onx.SerializeToString())

  You can load the model in Python and use the `ONNX` runtime to get
  predictions::

      from onnxruntime import InferenceSession
      with open("filename.onnx", "rb") as f:
          onx = f.read()
      sess = InferenceSession(onx, providers=["CPUExecutionProvider"])
      pred_ort = sess.run(None, {"X": X_test.astype(numpy.float32)})[0]

.. _skops_persistence:

`skops.io`
----------

:mod:`skops.io` avoids using :mod:`pickle` and only loads files which have types
and references to functions which are trusted either by default or by the user.
Therefore it provides a more secure format than :mod:`pickle`, :mod:`joblib`,
and `cloudpickle`_.

.. dropdown:: Using skops

  The API is very similar to :mod:`pickle`, and you can persist your models as
  explained in the `documentation
  <https://skops.readthedocs.io/en/stable/persistence.html>`__ using
  :func:`skops.io.dump` and :func:`skops.io.dumps`::

      import skops.io as sio
      obj = sio.dump(clf, "filename.skops")

  And you can load them back using :func:`skops.io.load` and
  :func:`skops.io.loads`. However, you need to specify the types which are
  trusted by you. You can get existing unknown types in a dumped object / file
  using :func:`skops.io.get_untrusted_types`, and after checking its contents,
  pass it to the load function::

      unknown_types = sio.get_untrusted_types(file="filename.skops")
      # investigate the contents of unknown_types, and only load if you trust
      # everything you see.
      clf = sio.load("filename.skops", trusted=unknown_types)

  Please report issues and feature requests related to this format on the `skops
  issue tracker <https://github.com/skops-dev/skops/issues>`__.

.. _pickle_persistence:

`pickle`, `joblib`, and `cloudpickle`
-------------------------------------

These three modules / packages, use the `pickle` protocol under the hood, but
come with slight variations:

- :mod:`pickle` is a module from the Python Standard Library. It can serialize
  and  deserialize any Python object, including custom Python classes and
  objects.
- :mod:`joblib` is more efficient than `pickle` when working with large machine
  learning models or large numpy arrays.
- `cloudpickle`_ can serialize certain objects which cannot be serialized by
  :mod:`pickle` or :mod:`joblib`, such as user defined functions and lambda
  functions. This can happen for instance, when using a
  :class:`~sklearn.preprocessing.FunctionTransformer` and using a custom
  function to transform the data.

.. dropdown:: Using `pickle`, `joblib`, or `cloudpickle`

  Depending on your use-case, you can choose one of these three methods to
  persist and load your scikit-learn model, and they all follow the same API::

      # Here you can replace pickle with joblib or cloudpickle
      from pickle import dump
      with open("filename.pkl", "wb") as f:
          dump(clf, f, protocol=5)

  Using `protocol=5` is recommended to reduce memory usage and make it faster to
  store and load any large NumPy array stored as a fitted attribute in the model.
  You can alternatively pass `protocol=pickle.HIGHEST_PROTOCOL` which is
  equivalent to `protocol=5` in Python 3.8 and later (at the time of writing).

  And later when needed, you can load the same object from the persisted file::

      # Here you can replace pickle with joblib or cloudpickle
      from pickle import load
      with open("filename.pkl", "rb") as f:
          clf = load(f)

.. _persistence_limitations:

Security & Maintainability Limitations
--------------------------------------

:mod:`pickle` (and :mod:`joblib` and :mod:`clouldpickle` by extension), has
many documented security vulnerabilities by design and should only be used if
the artifact, i.e. the pickle-file, is coming from a trusted and verified
source. You should never load a pickle file from an untrusted source, similarly
to how you should never execute code from an untrusted source.

Also note that arbitrary computations can be represented using the `ONNX`
format, and it is therefore recommended to serve models using `ONNX` in a
sandboxed environment to safeguard against computational and memory exploits.

Also note that there are no supported ways to load a model trained with a
different version of scikit-learn. While using :mod:`skops.io`, :mod:`joblib`,
:mod:`pickle`, or `cloudpickle`_, models saved using one version of
scikit-learn might load in other versions, however, this is entirely
unsupported and inadvisable. It should also be kept in mind that operations
performed on such data could give different and unexpected results, or even
crash your Python process.

In order to rebuild a similar model with future versions of scikit-learn,
additional metadata should be saved along the pickled model:

* The training data, e.g. a reference to an immutable snapshot
* The Python source code used to generate the model
* The versions of scikit-learn and its dependencies
* The cross validation score obtained on the training data

This should make it possible to check that the cross-validation score is in the
same range as before.

Aside for a few exceptions, persisted models should be portable across
operating systems and hardware architectures assuming the same versions of
dependencies and Python are used. If you encounter an estimator that is not
portable, please open an issue on GitHub. Persisted models are often deployed
in production using containers like Docker, in order to freeze the environment
and dependencies.

If you want to know more about these issues, please refer to these talks:

- `Adrin Jalali: Let's exploit pickle, and skops to the rescue! | PyData
  Amsterdam 2023 <https://www.youtube.com/watch?v=9w_H5OSTO9A>`__.
- `Alex Gaynor: Pickles are for Delis, not Software - PyCon 2014
  <https://pyvideo.org/video/2566/pickles-are-for-delis-not-software>`__.

.. _serving_environment:

Replicating the training environment in production
..................................................

If the versions of the dependencies used may differ from training to
production, it may result in unexpected behaviour and errors while using the
trained model. To prevent such situations it is recommended to use the same
dependencies and versions in both the training and production environment.
These transitive dependencies can be pinned with the help of package management
tools like `pip`, `mamba`, `conda`, `poetry`, `conda-lock`, `pixi`, etc.

It is not always possible to load an model trained with older versions of the
scikit-learn library and its dependencies in an updated software environment.
Instead, you might need to retrain the model with the new versions of the all
the libraries. So when training a model, it is important to record the training
recipe (e.g. a Python script) and training set information, and metadata about
all the dependencies to be able to automatically reconstruct the same training
environment for the updated software.

.. dropdown:: InconsistentVersionWarning

  When an estimator is loaded with a scikit-learn version that is inconsistent
  with the version the estimator was pickled with, a
  :class:`~sklearn.exceptions.InconsistentVersionWarning` is raised. This warning
  can be caught to obtain the original version the estimator was pickled with::

    from sklearn.exceptions import InconsistentVersionWarning
    warnings.simplefilter("error", InconsistentVersionWarning)

    try:
        with open("model_from_prevision_version.pickle", "rb") as f:
            est = pickle.load(f)
    except InconsistentVersionWarning as w:
        print(w.original_sklearn_version)

Serving the model artifact
..........................

The last step after training a scikit-learn model is serving the model.
Once the trained model is successfully loaded, it can be served to manage
different prediction requests. This can involve deploying the model as a
web service using containerization, or other model deployment strategies,
according to the specifications.

Summarizing the key points
--------------------------

Based on the different approaches for model persistence, the key points for
each approach can be summarized as follows:

* `ONNX`: It provides a uniform format for persisting any machine learning or
  deep learning model (other than scikit-learn) and is useful for model
  inference (predictions). It can however, result in compatibility issues with
  different frameworks.
* :mod:`skops.io`: Trained scikit-learn models can be easily shared and put
  into production using :mod:`skops.io`. It is more secure compared to
  alternate approaches based on :mod:`pickle` because it does not load
  arbitrary code unless explicitly asked for by the user. Such code needs to be
  packaged and importable in the target Python environment.
* :mod:`joblib`: Efficient memory mapping techniques make it faster when using
  the same persisted model in multiple Python processes when using
  `mmap_mode="r"`. It also gives easy shortcuts to compress and decompress the
  persisted object without the need for extra code. However, it may trigger the
  execution of malicious code when loading a model from an untrusted source as
  any other pickle-based persistence mechanism.
* :mod:`pickle`: It is native to Python and most Python objects can be
  serialized and deserialized using :mod:`pickle`, including custom Python
  classes and functions as long as they are defined in a package that can be
  imported in the target environment. While :mod:`pickle` can be used to easily
  save and load scikit-learn models, it may trigger the execution of malicious
  code while loading a model from an untrusted source. :mod:`pickle` can also
  be very efficient memorywise if the model was persisted with `protocol=5` but
  it does not support memory mapping.
* `cloudpickle`_: It has comparable loading efficiency as :mod:`pickle` and
  :mod:`joblib` (without memory mapping), but offers additional flexibility to
  serialize custom Python code such as lambda expressions and interactively
  defined functions and classes. It might be a last resort to persist pipelines
  with custom Python components such as a
  :class:`sklearn.preprocessing.FunctionTransformer` that wraps a function
  defined in the training script itself or more generally outside of any
  importable Python package. Note that `cloudpickle`_ offers no forward
  compatibility guarantees and you might need the same version of
  `cloudpickle`_ to load the persisted model along with the same version of all
  the libraries used to define the model. As the other pickle-based persistence
  mechanisms, it may trigger the execution of malicious code while loading
  a model from an untrusted source.

.. _cloudpickle: https://github.com/cloudpipe/cloudpickle

model persistence                     Model persistence                       list table   Summary of model persistence methods     widths  25 50 50     header rows  1         Persistence method        Pros        Risks   Cons         ref  ONNX  onnx persistence            Serve models without a Python environment          Serving and training environments independent of one another          Most secure option          Not all scikit learn models are supported          Custom estimators require more work to support          Original Python object is lost and cannot be reconstructed         ref  skops persistence           More secure than  pickle  based formats          Contents can be partly validated without loading          Not as fast as  pickle  based formats          Supports less types than  pickle  based formats          Requires the same environment as the training environment         mod  pickle           Native to Python          Can serialize most Python objects          Efficient memory usage with  protocol 5           Loading can execute arbitrary code          Requires the same environment as the training environment         mod  joblib           Efficient memory usage          Supports memory mapping          Easy shortcuts for compression and decompression          Pickle based format          Loading can execute arbitrary code          Requires the same environment as the training environment         cloudpickle            Can serialize non packaged  custom Python code          Comparable loading efficiency as  mod  pickle  with  protocol 5           Pickle based format          Loading can execute arbitrary code          No forward compatibility guarantees          Requires the same environment as the training environment  After training a scikit learn model  it is desirable to have a way to persist the model for future use without having to retrain  Based on your use case  there are a few different ways to persist a scikit learn model  and here we help you decide which one suits you best  In order to make a decision  you need to answer the following questions   1  Do you need the Python object after persistence  or do you only need to    persist in order to serve the model and get predictions out of it   If you only need to serve the model and no further investigation on the Python object itself is required  then  ref  ONNX  onnx persistence   might be the best fit for you  Note that not all models are supported by ONNX   In case ONNX is not suitable for your use case  the next question is   2  Do you absolutely trust the source of the model  or are there any security    concerns regarding where the persisted model comes from   If you have security concerns  then you should consider using  ref  skops io  skops persistence   which gives you back the Python object  but unlike  pickle  based persistence solutions  loading the persisted model doesn t automatically allow arbitrary code execution  Note that this requires manual investigation of the persisted file  which  mod  skops io  allows you to do   The other solutions assume you absolutely trust the source of the file to be loaded  as they are all susceptible to arbitrary code execution upon loading the persisted file since they all use the pickle protocol under the hood   3  Do you care about the performance of loading the model  and sharing it    between processes where a memory mapped object on disk is beneficial   If yes  then you can consider using  ref  joblib  pickle persistence    If this is not a major concern for you  then you can use the built in  mod  pickle  module   4  Did you try  mod  pickle  or  mod  joblib  and found that the model cannot    be persisted  It can happen for instance when you have user defined    functions in your model   If yes  then you can use  cloudpickle   which can serialize certain objects which cannot be serialized by  mod  pickle  or  mod  joblib     Workflow Overview                    In a typical workflow  the first step is to train the model using scikit learn and scikit learn compatible libraries  Note that support for scikit learn and third party estimators varies across the different persistence methods   Train and Persist the Model                              Creating an appropriate model depends on your use case  As an example  here we train a  class  sklearn ensemble HistGradientBoostingClassifier  on the iris dataset          from sklearn import ensemble       from sklearn import datasets       clf   ensemble HistGradientBoostingClassifier         X  y   datasets load iris return X y True        clf fit X  y    HistGradientBoostingClassifier    Once the model is trained  you can persist it using your desired method  and then you can load the model in a separate environment and get predictions from it given input data  Here there are two major paths depending on how you persist and plan to serve the model      ref  ONNX  onnx persistence    You need an  ONNX  runtime and an environment   with appropriate dependencies installed to load the model and use the runtime   to get predictions  This environment can be minimal and does not necessarily   even require Python to be installed to load the model and compute   predictions  Also note that  onnxruntime  typically requires much less RAM   than Python to compute predictions from small models      mod  skops io    mod  pickle    mod  joblib    cloudpickle    You need a   Python environment with the appropriate dependencies installed to load the   model and get predictions from it  This environment should have the same     packages   and the same   versions   as the environment where the model was   trained  Note that none of these methods support loading a model trained with   a different version of scikit learn  and possibly different versions of other   dependencies such as  numpy  and  scipy   Another concern would be running   the persisted model on a different hardware  and in most cases you should be   able to load your persisted model on a different hardware        onnx persistence   ONNX        ONNX   or  Open Neural Network Exchange  https   onnx ai      format is best suitable in use cases where one needs to persist the model and then use the persisted artifact to get predictions without the need to load the Python object itself  It is also useful in cases where the serving environment needs to be lean and minimal  since the  ONNX  runtime does not require  python     ONNX  is a binary serialization of the model  It has been developed to improve the usability of the interoperable representation of data models  It aims to facilitate the conversion of the data models between different machine learning frameworks  and to improve their portability on different computing architectures  More details are available from the  ONNX tutorial  https   onnx ai get started html      To convert scikit learn model to  ONNX   sklearn onnx  http   onnx ai sklearn onnx      has been developed  However  not all scikit learn models are supported  and it is limited to the core scikit learn and does not support most third party estimators  One can write a custom converter for third party or custom estimators  but the documentation to do that is sparse and it might be challenging to do so      dropdown   Using ONNX    To convert the model to  ONNX  format  you need to give the converter some   information about the input as well  about which you can read more  here    http   onnx ai sklearn onnx index html              from skl2onnx import to onnx       onx   to onnx clf  X  1  astype numpy float32   target opset 12        with open  filename onnx    wb   as f            f write onx SerializeToString       You can load the model in Python and use the  ONNX  runtime to get   predictions          from onnxruntime import InferenceSession       with open  filename onnx    rb   as f            onx   f read         sess   InferenceSession onx  providers   CPUExecutionProvider          pred ort   sess run None    X   X test astype numpy float32    0       skops persistence    skops io               mod  skops io  avoids using  mod  pickle  and only loads files which have types and references to functions which are trusted either by default or by the user  Therefore it provides a more secure format than  mod  pickle    mod  joblib   and  cloudpickle         dropdown   Using skops    The API is very similar to  mod  pickle   and you can persist your models as   explained in the  documentation    https   skops readthedocs io en stable persistence html     using    func  skops io dump  and  func  skops io dumps           import skops io as sio       obj   sio dump clf   filename skops      And you can load them back using  func  skops io load  and    func  skops io loads   However  you need to specify the types which are   trusted by you  You can get existing unknown types in a dumped object   file   using  func  skops io get untrusted types   and after checking its contents    pass it to the load function          unknown types   sio get untrusted types file  filename skops           investigate the contents of unknown types  and only load if you trust         everything you see        clf   sio load  filename skops   trusted unknown types     Please report issues and feature requests related to this format on the  skops   issue tracker  https   github com skops dev skops issues            pickle persistence    pickle    joblib   and  cloudpickle                                         These three modules   packages  use the  pickle  protocol under the hood  but come with slight variations      mod  pickle  is a module from the Python Standard Library  It can serialize   and  deserialize any Python object  including custom Python classes and   objects     mod  joblib  is more efficient than  pickle  when working with large machine   learning models or large numpy arrays     cloudpickle   can serialize certain objects which cannot be serialized by    mod  pickle  or  mod  joblib   such as user defined functions and lambda   functions  This can happen for instance  when using a    class   sklearn preprocessing FunctionTransformer  and using a custom   function to transform the data      dropdown   Using  pickle    joblib   or  cloudpickle     Depending on your use case  you can choose one of these three methods to   persist and load your scikit learn model  and they all follow the same API            Here you can replace pickle with joblib or cloudpickle       from pickle import dump       with open  filename pkl    wb   as f            dump clf  f  protocol 5     Using  protocol 5  is recommended to reduce memory usage and make it faster to   store and load any large NumPy array stored as a fitted attribute in the model    You can alternatively pass  protocol pickle HIGHEST PROTOCOL  which is   equivalent to  protocol 5  in Python 3 8 and later  at the time of writing      And later when needed  you can load the same object from the persisted file            Here you can replace pickle with joblib or cloudpickle       from pickle import load       with open  filename pkl    rb   as f            clf   load f       persistence limitations   Security   Maintainability Limitations                                          mod  pickle   and  mod  joblib  and  mod  clouldpickle  by extension   has many documented security vulnerabilities by design and should only be used if the artifact  i e  the pickle file  is coming from a trusted and verified source  You should never load a pickle file from an untrusted source  similarly to how you should never execute code from an untrusted source   Also note that arbitrary computations can be represented using the  ONNX  format  and it is therefore recommended to serve models using  ONNX  in a sandboxed environment to safeguard against computational and memory exploits   Also note that there are no supported ways to load a model trained with a different version of scikit learn  While using  mod  skops io    mod  joblib    mod  pickle   or  cloudpickle    models saved using one version of scikit learn might load in other versions  however  this is entirely unsupported and inadvisable  It should also be kept in mind that operations performed on such data could give different and unexpected results  or even crash your Python process   In order to rebuild a similar model with future versions of scikit learn  additional metadata should be saved along the pickled model     The training data  e g  a reference to an immutable snapshot   The Python source code used to generate the model   The versions of scikit learn and its dependencies   The cross validation score obtained on the training data  This should make it possible to check that the cross validation score is in the same range as before   Aside for a few exceptions  persisted models should be portable across operating systems and hardware architectures assuming the same versions of dependencies and Python are used  If you encounter an estimator that is not portable  please open an issue on GitHub  Persisted models are often deployed in production using containers like Docker  in order to freeze the environment and dependencies   If you want to know more about these issues  please refer to these talks      Adrin Jalali  Let s exploit pickle  and skops to the rescue    PyData   Amsterdam 2023  https   www youtube com watch v 9w H5OSTO9A         Alex Gaynor  Pickles are for Delis  not Software   PyCon 2014    https   pyvideo org video 2566 pickles are for delis not software            serving environment   Replicating the training environment in production                                                     If the versions of the dependencies used may differ from training to production  it may result in unexpected behaviour and errors while using the trained model  To prevent such situations it is recommended to use the same dependencies and versions in both the training and production environment  These transitive dependencies can be pinned with the help of package management tools like  pip    mamba    conda    poetry    conda lock    pixi   etc   It is not always possible to load an model trained with older versions of the scikit learn library and its dependencies in an updated software environment  Instead  you might need to retrain the model with the new versions of the all the libraries  So when training a model  it is important to record the training recipe  e g  a Python script  and training set information  and metadata about all the dependencies to be able to automatically reconstruct the same training environment for the updated software      dropdown   InconsistentVersionWarning    When an estimator is loaded with a scikit learn version that is inconsistent   with the version the estimator was pickled with  a    class   sklearn exceptions InconsistentVersionWarning  is raised  This warning   can be caught to obtain the original version the estimator was pickled with        from sklearn exceptions import InconsistentVersionWarning     warnings simplefilter  error   InconsistentVersionWarning       try          with open  model from prevision version pickle    rb   as f              est   pickle load f      except InconsistentVersionWarning as w          print w original sklearn version    Serving the model artifact                             The last step after training a scikit learn model is serving the model  Once the trained model is successfully loaded  it can be served to manage different prediction requests  This can involve deploying the model as a web service using containerization  or other model deployment strategies  according to the specifications    Summarizing the key points                             Based on the different approaches for model persistence  the key points for each approach can be summarized as follows      ONNX   It provides a uniform format for persisting any machine learning or   deep learning model  other than scikit learn  and is useful for model   inference  predictions   It can however  result in compatibility issues with   different frameworks     mod  skops io   Trained scikit learn models can be easily shared and put   into production using  mod  skops io   It is more secure compared to   alternate approaches based on  mod  pickle  because it does not load   arbitrary code unless explicitly asked for by the user  Such code needs to be   packaged and importable in the target Python environment     mod  joblib   Efficient memory mapping techniques make it faster when using   the same persisted model in multiple Python processes when using    mmap mode  r    It also gives easy shortcuts to compress and decompress the   persisted object without the need for extra code  However  it may trigger the   execution of malicious code when loading a model from an untrusted source as   any other pickle based persistence mechanism     mod  pickle   It is native to Python and most Python objects can be   serialized and deserialized using  mod  pickle   including custom Python   classes and functions as long as they are defined in a package that can be   imported in the target environment  While  mod  pickle  can be used to easily   save and load scikit learn models  it may trigger the execution of malicious   code while loading a model from an untrusted source   mod  pickle  can also   be very efficient memorywise if the model was persisted with  protocol 5  but   it does not support memory mapping     cloudpickle    It has comparable loading efficiency as  mod  pickle  and    mod  joblib   without memory mapping   but offers additional flexibility to   serialize custom Python code such as lambda expressions and interactively   defined functions and classes  It might be a last resort to persist pipelines   with custom Python components such as a    class  sklearn preprocessing FunctionTransformer  that wraps a function   defined in the training script itself or more generally outside of any   importable Python package  Note that  cloudpickle   offers no forward   compatibility guarantees and you might need the same version of    cloudpickle   to load the persisted model along with the same version of all   the libraries used to define the model  As the other pickle based persistence   mechanisms  it may trigger the execution of malicious code while loading   a model from an untrusted source       cloudpickle  https   github com cloudpipe cloudpickle


scikit-learn governance Scikit learn governance and decision making elements of our community interact scikit learn project to clarify how decisions are made and how the various This document establishes a decision making structure that takes into account The purpose of this document is to formalize the governance process used by the

Answer:
.. _governance:

===========================================
Scikit-learn governance and decision-making
===========================================

The purpose of this document is to formalize the governance process used by the
scikit-learn project, to clarify how decisions are made and how the various
elements of our community interact.
This document establishes a decision-making structure that takes into account
feedback from all members of the community and strives to find consensus, while
avoiding any deadlocks.

This is a meritocratic, consensus-based community project. Anyone with an
interest in the project can join the community, contribute to the project
design and participate in the decision making process. This document describes
how that participation takes place and how to set about earning merit within
the project community.

Roles And Responsibilities
==========================

We distinguish between contributors, core contributors, and the technical
committee. A key distinction between them is their voting rights: contributors
have no voting rights, whereas the other two groups all have voting rights,
as well as permissions to the tools relevant to their roles.

Contributors
------------

Contributors are community members who contribute in concrete ways to the
project. Anyone can become a contributor, and contributions can take many forms
– not only code – as detailed in the :ref:`contributors guide <contributing>`.
There is no process to become a contributor: once somebody contributes to the
project in any way, they are a contributor.

Core Contributors
-----------------

All core contributor members have the same voting rights and right to propose
new members to any of the roles listed below. Their membership is represented
as being an organization member on the scikit-learn `GitHub organization
<https://github.com/orgs/scikit-learn/people>`_.

They are also welcome to join our `monthly core contributor meetings
<https://github.com/scikit-learn/administrative/tree/master/meeting_notes>`_.

New members can be nominated by any existing member. Once they have been
nominated, there will be a vote by the current core contributors. Voting on new
members is one of the few activities that takes place on the project's private
mailing list. While it is expected that most votes will be unanimous, a
two-thirds majority of the cast votes is enough. The vote needs to be open for
at least 1 week.

Core contributors that have not contributed to the project, corresponding to
their role, in the past 12 months will be asked if they want to become emeritus
members and recant their rights until they become active again. The list of
members, active and emeritus (with dates at which they became active) is public
on the scikit-learn website. It is the responsibility of the active core
contributors to send such a yearly reminder email.

The following teams form the core contributors group:

* **Contributor Experience Team**
  The contributor experience team improves the experience of contributors by
  helping with the triage of issues and pull requests, as well as noticing any
  repeating patterns where people might struggle, and to help with improving
  those aspects of the project.

  To this end, they have the required permissions on GitHub to label and close
  issues. :ref:`Their work <bug_triaging>` is crucial to improve the
  communication in the project and limit the crowding of the issue tracker.

  .. _communication_team:

* **Communication Team**
  Members of the communication team help with outreach and communication
  for scikit-learn. The goal of the team is to develop public awareness of
  scikit-learn, of its features and usage, as well as branding.

  For this, they can operate the scikit-learn accounts on various social networks
  and produce materials. They also have the required rights to our blog
  repository and other relevant accounts and platforms.

* **Documentation Team**
  Members of the documentation team engage with the documentation of the project
  among other things. They might also be involved in other aspects of the
  project, but their reviews on documentation contributions are considered
  authoritative, and can merge such contributions.

  To this end, they have permissions to merge pull requests in scikit-learn's
  repository.

* **Maintainers Team**
  Maintainers are community members who have shown that they are dedicated to the
  continued development of the project through ongoing engagement with the
  community. They have shown they can be trusted to maintain scikit-learn with
  care. Being a maintainer allows contributors to more easily carry on with their
  project related activities by giving them direct access to the project's
  repository. Maintainers are expected to review code contributions, merge
  approved pull requests, cast votes for and against merging a pull-request,
  and to be involved in deciding major changes to the API.

Technical Committee
-------------------

The Technical Committee (TC) members are maintainers who have additional
responsibilities to ensure the smooth running of the project. TC members are
expected to participate in strategic planning, and approve changes to the
governance model. The purpose of the TC is to ensure a smooth progress from the
big-picture perspective. Indeed changes that impact the full project require a
synthetic analysis and a consensus that is both explicit and informed. In cases
that the core contributor community (which includes the TC members) fails to
reach such a consensus in the required time frame, the TC is the entity to
resolve the issue. Membership of the TC is by nomination by a core contributor.
A nomination will result in discussion which cannot take more than a month and
then a vote by the core contributors which will stay open for a week. TC
membership votes are subject to a two-third majority of all cast votes as well
as a simple majority approval of all the current TC members. TC members who do
not actively engage with the TC duties are expected to resign.

The Technical Committee of scikit-learn consists of :user:`Thomas Fan
<thomasjpfan>`, :user:`Alexandre Gramfort <agramfort>`, :user:`Olivier Grisel
<ogrisel>`, :user:`Adrin Jalali <adrinjalali>`, :user:`Andreas Müller
<amueller>`, :user:`Joel Nothman <jnothman>` and :user:`Gaël Varoquaux
<GaelVaroquaux>`.

Decision Making Process
=======================
Decisions about the future of the project are made through discussion with all
members of the community. All non-sensitive project management discussion takes
place on the project contributors' `mailing list <mailto:scikit-learn@python.org>`_
and the `issue tracker <https://github.com/scikit-learn/scikit-learn/issues>`_.
Occasionally, sensitive discussion occurs on a private list.

Scikit-learn uses a "consensus seeking" process for making decisions. The group
tries to find a resolution that has no open objections among core contributors.
At any point during the discussion, any core contributor can call for a vote,
which will conclude one month from the call for the vote. Most votes have to be
backed by a :ref:`SLEP <slep>`. If no option can gather two thirds of the votes
cast, the decision is escalated to the TC, which in turn will use consensus
seeking with the fallback option of a simple majority vote if no consensus can
be found within a month. This is what we hereafter may refer to as "**the
decision making process**".

Decisions (in addition to adding core contributors and TC membership as above)
are made according to the following rules:

* **Minor Documentation changes**, such as typo fixes, or addition / correction
  of a sentence, but no change of the ``scikit-learn.org`` landing page or the
  “about” page: Requires +1 by a maintainer, no -1 by a maintainer (lazy
  consensus), happens on the issue or pull request page. Maintainers are
  expected to give “reasonable time” to others to give their opinion on the
  pull request if they're not confident others would agree.

* **Code changes and major documentation changes**
  require +1 by two maintainers, no -1 by a maintainer (lazy
  consensus), happens on the issue of pull-request page.

* **Changes to the API principles and changes to dependencies or supported
  versions** happen via :ref:`slep` and follows the decision-making process
  outlined above.

* **Changes to the governance model** follow the process outlined in `SLEP020
  <https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep020/proposal.html>`__.

If a veto -1 vote is cast on a lazy consensus, the proposer can appeal to the
community and maintainers and the change can be approved or rejected using
the decision making procedure outlined above.

Governance Model Changes
------------------------

Governance model changes occur through an enhancement proposal or a GitHub Pull
Request. An enhancement proposal will go through "**the decision-making process**"
described in the previous section. Alternatively, an author may propose a change
directly to the governance model with a GitHub Pull Request. Logistically, an
author can open a Draft Pull Request for feedback and follow up with a new
revised Pull Request for voting. Once that author is happy with the state of the
Pull Request, they can call for a vote on the public mailing list. During the
one-month voting period, the Pull Request can not change. A Pull Request
Approval will count as a positive vote, and a "Request Changes" review will
count as a negative vote. If two-thirds of the cast votes are positive, then
the governance model change is accepted.

.. _slep:

Enhancement proposals (SLEPs)
==============================
For all votes, a proposal must have been made public and discussed before the
vote. Such proposal must be a consolidated document, in the form of a
"Scikit-Learn Enhancement Proposal" (SLEP), rather than a long discussion on an
issue. A SLEP must be submitted as a pull-request to `enhancement proposals
<https://scikit-learn-enhancement-proposals.readthedocs.io>`_ using the `SLEP
template
<https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep_template.html>`_.
`SLEP000
<https://scikit-learn-enhancement-proposals.readthedocs.io/en/latest/slep000/proposal.html>`__
describes the process in more detail.

governance                                               Scikit learn governance and decision making                                              The purpose of this document is to formalize the governance process used by the scikit learn project  to clarify how decisions are made and how the various elements of our community interact  This document establishes a decision making structure that takes into account feedback from all members of the community and strives to find consensus  while avoiding any deadlocks   This is a meritocratic  consensus based community project  Anyone with an interest in the project can join the community  contribute to the project design and participate in the decision making process  This document describes how that participation takes place and how to set about earning merit within the project community   Roles And Responsibilities                             We distinguish between contributors  core contributors  and the technical committee  A key distinction between them is their voting rights  contributors have no voting rights  whereas the other two groups all have voting rights  as well as permissions to the tools relevant to their roles   Contributors               Contributors are community members who contribute in concrete ways to the project  Anyone can become a contributor  and contributions can take many forms   not only code   as detailed in the  ref  contributors guide  contributing    There is no process to become a contributor  once somebody contributes to the project in any way  they are a contributor   Core Contributors                    All core contributor members have the same voting rights and right to propose new members to any of the roles listed below  Their membership is represented as being an organization member on the scikit learn  GitHub organization  https   github com orgs scikit learn people      They are also welcome to join our  monthly core contributor meetings  https   github com scikit learn administrative tree master meeting notes      New members can be nominated by any existing member  Once they have been nominated  there will be a vote by the current core contributors  Voting on new members is one of the few activities that takes place on the project s private mailing list  While it is expected that most votes will be unanimous  a two thirds majority of the cast votes is enough  The vote needs to be open for at least 1 week   Core contributors that have not contributed to the project  corresponding to their role  in the past 12 months will be asked if they want to become emeritus members and recant their rights until they become active again  The list of members  active and emeritus  with dates at which they became active  is public on the scikit learn website  It is the responsibility of the active core contributors to send such a yearly reminder email   The following teams form the core contributors group       Contributor Experience Team     The contributor experience team improves the experience of contributors by   helping with the triage of issues and pull requests  as well as noticing any   repeating patterns where people might struggle  and to help with improving   those aspects of the project     To this end  they have the required permissions on GitHub to label and close   issues   ref  Their work  bug triaging   is crucial to improve the   communication in the project and limit the crowding of the issue tracker         communication team       Communication Team     Members of the communication team help with outreach and communication   for scikit learn  The goal of the team is to develop public awareness of   scikit learn  of its features and usage  as well as branding     For this  they can operate the scikit learn accounts on various social networks   and produce materials  They also have the required rights to our blog   repository and other relevant accounts and platforms       Documentation Team     Members of the documentation team engage with the documentation of the project   among other things  They might also be involved in other aspects of the   project  but their reviews on documentation contributions are considered   authoritative  and can merge such contributions     To this end  they have permissions to merge pull requests in scikit learn s   repository       Maintainers Team     Maintainers are community members who have shown that they are dedicated to the   continued development of the project through ongoing engagement with the   community  They have shown they can be trusted to maintain scikit learn with   care  Being a maintainer allows contributors to more easily carry on with their   project related activities by giving them direct access to the project s   repository  Maintainers are expected to review code contributions  merge   approved pull requests  cast votes for and against merging a pull request    and to be involved in deciding major changes to the API   Technical Committee                      The Technical Committee  TC  members are maintainers who have additional responsibilities to ensure the smooth running of the project  TC members are expected to participate in strategic planning  and approve changes to the governance model  The purpose of the TC is to ensure a smooth progress from the big picture perspective  Indeed changes that impact the full project require a synthetic analysis and a consensus that is both explicit and informed  In cases that the core contributor community  which includes the TC members  fails to reach such a consensus in the required time frame  the TC is the entity to resolve the issue  Membership of the TC is by nomination by a core contributor  A nomination will result in discussion which cannot take more than a month and then a vote by the core contributors which will stay open for a week  TC membership votes are subject to a two third majority of all cast votes as well as a simple majority approval of all the current TC members  TC members who do not actively engage with the TC duties are expected to resign   The Technical Committee of scikit learn consists of  user  Thomas Fan  thomasjpfan     user  Alexandre Gramfort  agramfort     user  Olivier Grisel  ogrisel     user  Adrin Jalali  adrinjalali     user  Andreas M ller  amueller     user  Joel Nothman  jnothman   and  user  Ga l Varoquaux  GaelVaroquaux     Decision Making Process                         Decisions about the future of the project are made through discussion with all members of the community  All non sensitive project management discussion takes place on the project contributors   mailing list  mailto scikit learn python org    and the  issue tracker  https   github com scikit learn scikit learn issues     Occasionally  sensitive discussion occurs on a private list   Scikit learn uses a  consensus seeking  process for making decisions  The group tries to find a resolution that has no open objections among core contributors  At any point during the discussion  any core contributor can call for a vote  which will conclude one month from the call for the vote  Most votes have to be backed by a  ref  SLEP  slep    If no option can gather two thirds of the votes cast  the decision is escalated to the TC  which in turn will use consensus seeking with the fallback option of a simple majority vote if no consensus can be found within a month  This is what we hereafter may refer to as    the decision making process      Decisions  in addition to adding core contributors and TC membership as above  are made according to the following rules       Minor Documentation changes    such as typo fixes  or addition   correction   of a sentence  but no change of the   scikit learn org   landing page or the    about  page  Requires  1 by a maintainer  no  1 by a maintainer  lazy   consensus   happens on the issue or pull request page  Maintainers are   expected to give  reasonable time  to others to give their opinion on the   pull request if they re not confident others would agree       Code changes and major documentation changes     require  1 by two maintainers  no  1 by a maintainer  lazy   consensus   happens on the issue of pull request page       Changes to the API principles and changes to dependencies or supported   versions   happen via  ref  slep  and follows the decision making process   outlined above       Changes to the governance model   follow the process outlined in  SLEP020    https   scikit learn enhancement proposals readthedocs io en latest slep020 proposal html       If a veto  1 vote is cast on a lazy consensus  the proposer can appeal to the community and maintainers and the change can be approved or rejected using the decision making procedure outlined above   Governance Model Changes                           Governance model changes occur through an enhancement proposal or a GitHub Pull Request  An enhancement proposal will go through    the decision making process    described in the previous section  Alternatively  an author may propose a change directly to the governance model with a GitHub Pull Request  Logistically  an author can open a Draft Pull Request for feedback and follow up with a new revised Pull Request for voting  Once that author is happy with the state of the Pull Request  they can call for a vote on the public mailing list  During the one month voting period  the Pull Request can not change  A Pull Request Approval will count as a positive vote  and a  Request Changes  review will count as a negative vote  If two thirds of the cast votes are positive  then the governance model change is accepted       slep   Enhancement proposals  SLEPs                                 For all votes  a proposal must have been made public and discussed before the vote  Such proposal must be a consolidated document  in the form of a  Scikit Learn Enhancement Proposal   SLEP   rather than a long discussion on an issue  A SLEP must be submitted as a pull request to  enhancement proposals  https   scikit learn enhancement proposals readthedocs io    using the  SLEP template  https   scikit learn enhancement proposals readthedocs io en latest slep template html      SLEP000  https   scikit learn enhancement proposals readthedocs io en latest slep000 proposal html     describes the process in more detail


scikit-learn Projects implementing the scikit learn estimator API are encouraged to use relatedprojects The which facilitates best practices for testing and documenting estimators Related Projects the

Answer:
.. _related_projects:

=====================================
Related Projects
=====================================

Projects implementing the scikit-learn estimator API are encouraged to use
the `scikit-learn-contrib template <https://github.com/scikit-learn-contrib/project-template>`_
which facilitates best practices for testing and documenting estimators.
The `scikit-learn-contrib GitHub organization <https://github.com/scikit-learn-contrib/scikit-learn-contrib>`_
also accepts high-quality contributions of repositories conforming to this
template.

Below is a list of sister-projects, extensions and domain specific packages.

Interoperability and framework enhancements
-------------------------------------------

These tools adapt scikit-learn for use with other technologies or otherwise
enhance the functionality of scikit-learn's estimators.

**Auto-ML**

- `auto-sklearn <https://github.com/automl/auto-sklearn/>`_
  An automated machine learning toolkit and a drop-in replacement for a
  scikit-learn estimator

- `autoviml <https://github.com/AutoViML/Auto_ViML/>`_
  Automatically Build Multiple Machine Learning Models with a Single Line of Code.
  Designed as a faster way to use scikit-learn models without having to preprocess data.

- `TPOT <https://github.com/rhiever/tpot>`_
  An automated machine learning toolkit that optimizes a series of scikit-learn
  operators to design a machine learning pipeline, including data and feature
  preprocessors as well as the estimators. Works as a drop-in replacement for a
  scikit-learn estimator.

- `Featuretools <https://github.com/alteryx/featuretools>`_
  A framework to perform automated feature engineering. It can be used for
  transforming temporal and relational datasets into feature matrices for
  machine learning.

- `EvalML <https://github.com/alteryx/evalml>`_
  EvalML is an AutoML library which builds, optimizes, and evaluates
  machine learning pipelines using domain-specific objective functions.
  It incorporates multiple modeling libraries under one API, and
  the objects that EvalML creates use an sklearn-compatible API.

- `MLJAR AutoML <https://github.com/mljar/mljar-supervised>`_
  Python package for AutoML on Tabular Data with Feature Engineering,
  Hyper-Parameters Tuning, Explanations and Automatic Documentation.

**Experimentation and model registry frameworks**

- `MLFlow <https://mlflow.org/>`_ MLflow is an open source platform to manage the ML
  lifecycle, including experimentation, reproducibility, deployment, and a central
  model registry.

- `Neptune <https://neptune.ai/>`_ Metadata store for MLOps,
  built for teams that run a lot of experiments. It gives you a single
  place to log, store, display, organize, compare, and query all your
  model building metadata.

- `Sacred <https://github.com/IDSIA/Sacred>`_ Tool to help you configure,
  organize, log and reproduce experiments

- `Scikit-Learn Laboratory
  <https://skll.readthedocs.io/en/latest/index.html>`_  A command-line
  wrapper around scikit-learn that makes it easy to run machine learning
  experiments with multiple learners and large feature sets.

**Model inspection and visualization**

- `dtreeviz <https://github.com/parrt/dtreeviz/>`_ A python library for
  decision tree visualization and model interpretation.

- `sklearn-evaluation <https://github.com/ploomber/sklearn-evaluation>`_
  Machine learning model evaluation made easy: plots, tables, HTML reports,
  experiment tracking and Jupyter notebook analysis. Visual analysis, model
  selection, evaluation and diagnostics.

- `yellowbrick <https://github.com/DistrictDataLabs/yellowbrick>`_ A suite of
  custom matplotlib visualizers for scikit-learn estimators to support visual feature
  analysis, model selection, evaluation, and diagnostics.

**Model export for production**

- `sklearn-onnx <https://github.com/onnx/sklearn-onnx>`_ Serialization of many
  Scikit-learn pipelines to `ONNX <https://onnx.ai/>`_ for interchange and
  prediction.

- `skops.io <https://skops.readthedocs.io/en/stable/persistence.html>`__ A
  persistence model more secure than pickle, which can be used instead of
  pickle in most common cases.

- `sklearn2pmml <https://github.com/jpmml/sklearn2pmml>`_
  Serialization of a wide variety of scikit-learn estimators and transformers
  into PMML with the help of `JPMML-SkLearn <https://github.com/jpmml/jpmml-sklearn>`_
  library.

- `treelite <https://treelite.readthedocs.io>`_
  Compiles tree-based ensemble models into C code for minimizing prediction
  latency.

- `emlearn <https://emlearn.org>`_
  Implements scikit-learn estimators in C99 for embedded devices and microcontrollers.
  Supports several classifier, regression and outlier detection models.

**Model throughput**

- `Intel(R) Extension for scikit-learn <https://github.com/intel/scikit-learn-intelex>`_
  Mostly on high end Intel(R) hardware, accelerates some scikit-learn models
  for both training and inference under certain circumstances. This project is
  maintained by Intel(R) and scikit-learn's maintainers are not involved in the
  development of this project. Also note that in some cases using the tools and
  estimators under ``scikit-learn-intelex`` would give different results than
  ``scikit-learn`` itself. If you encounter issues while using this project,
  make sure you report potential issues in their respective repositories.

**Interface to R with genomic applications**

- `BiocSklearn <https://bioconductor.org/packages/BiocSklearn>`_
  Exposes a small number of dimension reduction facilities as an illustration
  of the basilisk protocol for interfacing python with R. Intended as a
  springboard for more complete interop.

Other estimators and tasks
--------------------------

Not everything belongs or is mature enough for the central scikit-learn
project. The following are projects providing interfaces similar to
scikit-learn for additional learning algorithms, infrastructures
and tasks.

**Time series and forecasting**

- `Darts <https://unit8co.github.io/darts/>`_ Darts is a Python library for
  user-friendly forecasting and anomaly detection on time series. It contains a variety
  of models, from classics such as ARIMA to deep neural networks. The forecasting
  models can all be used in the same way, using fit() and predict() functions, similar
  to scikit-learn.

- `sktime <https://github.com/alan-turing-institute/sktime>`_ A scikit-learn compatible
  toolbox for machine learning with time series including time series
  classification/regression and (supervised/panel) forecasting.

- `skforecast <https://github.com/JoaquinAmatRodrigo/skforecast>`_ A python library
  that eases using scikit-learn regressors as multi-step forecasters. It also works
  with any regressor compatible with the scikit-learn API.

- `tslearn <https://github.com/tslearn-team/tslearn>`_ A machine learning library for
  time series that offers tools for pre-processing and feature extraction as well as
  dedicated models for clustering, classification and regression.

**Gradient (tree) boosting**

Note scikit-learn own modern gradient boosting estimators
:class:`~sklearn.ensemble.HistGradientBoostingClassifier` and
:class:`~sklearn.ensemble.HistGradientBoostingRegressor`.

- `XGBoost <https://github.com/dmlc/xgboost>`_ XGBoost is an optimized distributed
  gradient boosting library designed to be highly efficient, flexible and portable.

- `LightGBM <https://lightgbm.readthedocs.io>`_ LightGBM is a gradient boosting
  framework that uses tree based learning algorithms. It is designed to be distributed
  and efficient.

**Structured learning**

- `HMMLearn <https://github.com/hmmlearn/hmmlearn>`_ Implementation of hidden
  markov models that was previously part of scikit-learn.

- `pomegranate <https://github.com/jmschrei/pomegranate>`_ Probabilistic modelling
  for Python, with an emphasis on hidden Markov models.

**Deep neural networks etc.**

- `skorch <https://github.com/dnouri/skorch>`_ A scikit-learn compatible
  neural network library that wraps PyTorch.

- `scikeras <https://github.com/adriangb/scikeras>`_ provides a wrapper around
  Keras to interface it with scikit-learn. SciKeras is the successor
  of `tf.keras.wrappers.scikit_learn`.

**Federated Learning**

- `Flower <https://flower.dev/>`_ A friendly federated learning framework with a
  unified approach that can federate any workload, any ML framework, and any programming language.

**Privacy Preserving Machine Learning**

- `Concrete ML <https://github.com/zama-ai/concrete-ml/>`_ A privacy preserving
  ML framework built on top of `Concrete
  <https://github.com/zama-ai/concrete>`_, with bindings to traditional ML
  frameworks, thanks to fully homomorphic encryption. APIs of so-called
  Concrete ML built-in models are very close to scikit-learn APIs.

**Broad scope**

- `mlxtend <https://github.com/rasbt/mlxtend>`_ Includes a number of additional
  estimators as well as model visualization utilities.

- `scikit-lego <https://github.com/koaning/scikit-lego>`_ A number of scikit-learn compatible
  custom transformers, models and metrics, focusing on solving practical industry tasks.

**Other regression and classification**

- `py-earth <https://github.com/scikit-learn-contrib/py-earth>`_ Multivariate
  adaptive regression splines

- `gplearn <https://github.com/trevorstephens/gplearn>`_ Genetic Programming
  for symbolic regression tasks.

- `scikit-multilearn <https://github.com/scikit-multilearn/scikit-multilearn>`_
  Multi-label classification with focus on label space manipulation.

**Decomposition and clustering**

- `lda <https://github.com/lda-project/lda/>`_: Fast implementation of latent
  Dirichlet allocation in Cython which uses `Gibbs sampling
  <https://en.wikipedia.org/wiki/Gibbs_sampling>`_ to sample from the true
  posterior distribution. (scikit-learn's
  :class:`~sklearn.decomposition.LatentDirichletAllocation` implementation uses
  `variational inference
  <https://en.wikipedia.org/wiki/Variational_Bayesian_methods>`_ to sample from
  a tractable approximation of a topic model's posterior distribution.)

- `kmodes <https://github.com/nicodv/kmodes>`_ k-modes clustering algorithm for
  categorical data, and several of its variations.

- `hdbscan <https://github.com/scikit-learn-contrib/hdbscan>`_ HDBSCAN and Robust Single
  Linkage clustering algorithms for robust variable density clustering.
  As of scikit-learn version 1.3.0, there is :class:`~sklearn.cluster.HDBSCAN`.

**Pre-processing**

- `categorical-encoding
  <https://github.com/scikit-learn-contrib/categorical-encoding>`_ A
  library of sklearn compatible categorical variable encoders.
  As of scikit-learn version 1.3.0, there is
  :class:`~sklearn.preprocessing.TargetEncoder`.

- `imbalanced-learn
  <https://github.com/scikit-learn-contrib/imbalanced-learn>`_ Various
  methods to under- and over-sample datasets.

- `Feature-engine <https://github.com/solegalli/feature_engine>`_ A library
  of sklearn compatible transformers for missing data imputation, categorical
  encoding, variable transformation, discretization, outlier handling and more.
  Feature-engine allows the application of preprocessing steps to selected groups
  of variables and it is fully compatible with the Scikit-learn Pipeline.

**Topological Data Analysis**

- `giotto-tda <https://github.com/giotto-ai/giotto-tda>`_ A library for
  `Topological Data Analysis
  <https://en.wikipedia.org/wiki/Topological_data_analysis>`_ aiming to
  provide a scikit-learn compatible API. It offers tools to transform data
  inputs (point clouds, graphs, time series, images) into forms suitable for
  computations of topological summaries, and components dedicated to
  extracting sets of scalar features of topological origin, which can be used
  alongside other feature extraction methods in scikit-learn.

Statistical learning with Python
--------------------------------
Other packages useful for data analysis and machine learning.

- `Pandas <https://pandas.pydata.org/>`_ Tools for working with heterogeneous and
  columnar data, relational queries, time series and basic statistics.

- `statsmodels <https://www.statsmodels.org>`_ Estimating and analysing
  statistical models. More focused on statistical tests and less on prediction
  than scikit-learn.

- `PyMC <https://www.pymc.io/>`_ Bayesian statistical models and
  fitting algorithms.

- `Seaborn <https://stanford.edu/~mwaskom/software/seaborn/>`_ Visualization library based on
  matplotlib. It provides a high-level interface for drawing attractive statistical graphics.

- `scikit-survival <https://scikit-survival.readthedocs.io/>`_ A library implementing
  models to learn from censored time-to-event data (also called survival analysis).
  Models are fully compatible with scikit-learn.

Recommendation Engine packages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- `implicit <https://github.com/benfred/implicit>`_, Library for implicit
  feedback datasets.

- `lightfm <https://github.com/lyst/lightfm>`_ A Python/Cython
  implementation of a hybrid recommender system.

- `Surprise Lib <https://surpriselib.com/>`_ Library for explicit feedback
  datasets.

Domain specific packages
~~~~~~~~~~~~~~~~~~~~~~~~

- `scikit-network <https://scikit-network.readthedocs.io/>`_ Machine learning on graphs.

- `scikit-image <https://scikit-image.org/>`_ Image processing and computer
  vision in python.

- `Natural language toolkit (nltk) <https://www.nltk.org/>`_ Natural language
  processing and some machine learning.

- `gensim <https://radimrehurek.com/gensim/>`_  A library for topic modelling,
  document indexing and similarity retrieval

- `NiLearn <https://nilearn.github.io/>`_ Machine learning for neuro-imaging.

- `AstroML <https://www.astroml.org/>`_  Machine learning for astronomy.

Translations of scikit-learn documentation
------------------------------------------

Translation's purpose is to ease reading and understanding in languages
other than English. Its aim is to help people who do not understand English
or have doubts about its interpretation. Additionally, some people prefer
to read documentation in their native language, but please bear in mind that
the only official documentation is the English one [#f1]_.

Those translation efforts are community initiatives and we have no control
on them.
If you want to contribute or report an issue with the translation, please
contact the authors of the translation.
Some available translations are linked here to improve their dissemination
and promote community efforts.

- `Chinese translation <https://sklearn.apachecn.org/>`_
  (`source <https://github.com/apachecn/sklearn-doc-zh>`__)
- `Persian translation <https://sklearn.ir/>`_
  (`source <https://github.com/mehrdad-dev/scikit-learn>`__)
- `Spanish translation <https://qu4nt.github.io/sklearn-doc-es/>`_
  (`source <https://github.com/qu4nt/sklearn-doc-es>`__)
- `Korean translation <https://panda5176.github.io/scikit-learn-korean/>`_
  (`source <https://github.com/panda5176/scikit-learn-korean>`__)

.. rubric:: Footnotes

.. [#f1] following `linux documentation Disclaimer
   <https://www.kernel.org/doc/html/latest/translations/index.html#disclaimer>`__

related projects                                         Related Projects                                        Projects implementing the scikit learn estimator API are encouraged to use the  scikit learn contrib template  https   github com scikit learn contrib project template    which facilitates best practices for testing and documenting estimators  The  scikit learn contrib GitHub organization  https   github com scikit learn contrib scikit learn contrib    also accepts high quality contributions of repositories conforming to this template   Below is a list of sister projects  extensions and domain specific packages   Interoperability and framework enhancements                                              These tools adapt scikit learn for use with other technologies or otherwise enhance the functionality of scikit learn s estimators     Auto ML       auto sklearn  https   github com automl auto sklearn       An automated machine learning toolkit and a drop in replacement for a   scikit learn estimator     autoviml  https   github com AutoViML Auto ViML       Automatically Build Multiple Machine Learning Models with a Single Line of Code    Designed as a faster way to use scikit learn models without having to preprocess data      TPOT  https   github com rhiever tpot      An automated machine learning toolkit that optimizes a series of scikit learn   operators to design a machine learning pipeline  including data and feature   preprocessors as well as the estimators  Works as a drop in replacement for a   scikit learn estimator      Featuretools  https   github com alteryx featuretools      A framework to perform automated feature engineering  It can be used for   transforming temporal and relational datasets into feature matrices for   machine learning      EvalML  https   github com alteryx evalml      EvalML is an AutoML library which builds  optimizes  and evaluates   machine learning pipelines using domain specific objective functions    It incorporates multiple modeling libraries under one API  and   the objects that EvalML creates use an sklearn compatible API      MLJAR AutoML  https   github com mljar mljar supervised      Python package for AutoML on Tabular Data with Feature Engineering     Hyper Parameters Tuning  Explanations and Automatic Documentation     Experimentation and model registry frameworks       MLFlow  https   mlflow org     MLflow is an open source platform to manage the ML   lifecycle  including experimentation  reproducibility  deployment  and a central   model registry      Neptune  https   neptune ai     Metadata store for MLOps    built for teams that run a lot of experiments  It gives you a single   place to log  store  display  organize  compare  and query all your   model building metadata      Sacred  https   github com IDSIA Sacred    Tool to help you configure    organize  log and reproduce experiments     Scikit Learn Laboratory    https   skll readthedocs io en latest index html     A command line   wrapper around scikit learn that makes it easy to run machine learning   experiments with multiple learners and large feature sets     Model inspection and visualization       dtreeviz  https   github com parrt dtreeviz     A python library for   decision tree visualization and model interpretation      sklearn evaluation  https   github com ploomber sklearn evaluation      Machine learning model evaluation made easy  plots  tables  HTML reports    experiment tracking and Jupyter notebook analysis  Visual analysis  model   selection  evaluation and diagnostics      yellowbrick  https   github com DistrictDataLabs yellowbrick    A suite of   custom matplotlib visualizers for scikit learn estimators to support visual feature   analysis  model selection  evaluation  and diagnostics     Model export for production       sklearn onnx  https   github com onnx sklearn onnx    Serialization of many   Scikit learn pipelines to  ONNX  https   onnx ai     for interchange and   prediction      skops io  https   skops readthedocs io en stable persistence html     A   persistence model more secure than pickle  which can be used instead of   pickle in most common cases      sklearn2pmml  https   github com jpmml sklearn2pmml      Serialization of a wide variety of scikit learn estimators and transformers   into PMML with the help of  JPMML SkLearn  https   github com jpmml jpmml sklearn      library      treelite  https   treelite readthedocs io      Compiles tree based ensemble models into C code for minimizing prediction   latency      emlearn  https   emlearn org      Implements scikit learn estimators in C99 for embedded devices and microcontrollers    Supports several classifier  regression and outlier detection models     Model throughput       Intel R  Extension for scikit learn  https   github com intel scikit learn intelex      Mostly on high end Intel R  hardware  accelerates some scikit learn models   for both training and inference under certain circumstances  This project is   maintained by Intel R  and scikit learn s maintainers are not involved in the   development of this project  Also note that in some cases using the tools and   estimators under   scikit learn intelex   would give different results than     scikit learn   itself  If you encounter issues while using this project    make sure you report potential issues in their respective repositories     Interface to R with genomic applications       BiocSklearn  https   bioconductor org packages BiocSklearn      Exposes a small number of dimension reduction facilities as an illustration   of the basilisk protocol for interfacing python with R  Intended as a    springboard for more complete interop    Other estimators and tasks                             Not everything belongs or is mature enough for the central scikit learn project  The following are projects providing interfaces similar to scikit learn for additional learning algorithms  infrastructures and tasks     Time series and forecasting       Darts  https   unit8co github io darts     Darts is a Python library for   user friendly forecasting and anomaly detection on time series  It contains a variety   of models  from classics such as ARIMA to deep neural networks  The forecasting   models can all be used in the same way  using fit   and predict   functions  similar   to scikit learn      sktime  https   github com alan turing institute sktime    A scikit learn compatible   toolbox for machine learning with time series including time series   classification regression and  supervised panel  forecasting      skforecast  https   github com JoaquinAmatRodrigo skforecast    A python library   that eases using scikit learn regressors as multi step forecasters  It also works   with any regressor compatible with the scikit learn API      tslearn  https   github com tslearn team tslearn    A machine learning library for   time series that offers tools for pre processing and feature extraction as well as   dedicated models for clustering  classification and regression     Gradient  tree  boosting    Note scikit learn own modern gradient boosting estimators  class   sklearn ensemble HistGradientBoostingClassifier  and  class   sklearn ensemble HistGradientBoostingRegressor       XGBoost  https   github com dmlc xgboost    XGBoost is an optimized distributed   gradient boosting library designed to be highly efficient  flexible and portable      LightGBM  https   lightgbm readthedocs io    LightGBM is a gradient boosting   framework that uses tree based learning algorithms  It is designed to be distributed   and efficient     Structured learning       HMMLearn  https   github com hmmlearn hmmlearn    Implementation of hidden   markov models that was previously part of scikit learn      pomegranate  https   github com jmschrei pomegranate    Probabilistic modelling   for Python  with an emphasis on hidden Markov models     Deep neural networks etc        skorch  https   github com dnouri skorch    A scikit learn compatible   neural network library that wraps PyTorch      scikeras  https   github com adriangb scikeras    provides a wrapper around   Keras to interface it with scikit learn  SciKeras is the successor   of  tf keras wrappers scikit learn      Federated Learning       Flower  https   flower dev     A friendly federated learning framework with a   unified approach that can federate any workload  any ML framework  and any programming language     Privacy Preserving Machine Learning       Concrete ML  https   github com zama ai concrete ml     A privacy preserving   ML framework built on top of  Concrete    https   github com zama ai concrete     with bindings to traditional ML   frameworks  thanks to fully homomorphic encryption  APIs of so called   Concrete ML built in models are very close to scikit learn APIs     Broad scope       mlxtend  https   github com rasbt mlxtend    Includes a number of additional   estimators as well as model visualization utilities      scikit lego  https   github com koaning scikit lego    A number of scikit learn compatible   custom transformers  models and metrics  focusing on solving practical industry tasks     Other regression and classification       py earth  https   github com scikit learn contrib py earth    Multivariate   adaptive regression splines     gplearn  https   github com trevorstephens gplearn    Genetic Programming   for symbolic regression tasks      scikit multilearn  https   github com scikit multilearn scikit multilearn      Multi label classification with focus on label space manipulation     Decomposition and clustering       lda  https   github com lda project lda      Fast implementation of latent   Dirichlet allocation in Cython which uses  Gibbs sampling    https   en wikipedia org wiki Gibbs sampling    to sample from the true   posterior distribution   scikit learn s    class   sklearn decomposition LatentDirichletAllocation  implementation uses    variational inference    https   en wikipedia org wiki Variational Bayesian methods    to sample from   a tractable approximation of a topic model s posterior distribution       kmodes  https   github com nicodv kmodes    k modes clustering algorithm for   categorical data  and several of its variations      hdbscan  https   github com scikit learn contrib hdbscan    HDBSCAN and Robust Single   Linkage clustering algorithms for robust variable density clustering    As of scikit learn version 1 3 0  there is  class   sklearn cluster HDBSCAN      Pre processing       categorical encoding    https   github com scikit learn contrib categorical encoding    A   library of sklearn compatible categorical variable encoders    As of scikit learn version 1 3 0  there is    class   sklearn preprocessing TargetEncoder       imbalanced learn    https   github com scikit learn contrib imbalanced learn    Various   methods to under  and over sample datasets      Feature engine  https   github com solegalli feature engine    A library   of sklearn compatible transformers for missing data imputation  categorical   encoding  variable transformation  discretization  outlier handling and more    Feature engine allows the application of preprocessing steps to selected groups   of variables and it is fully compatible with the Scikit learn Pipeline     Topological Data Analysis       giotto tda  https   github com giotto ai giotto tda    A library for    Topological Data Analysis    https   en wikipedia org wiki Topological data analysis    aiming to   provide a scikit learn compatible API  It offers tools to transform data   inputs  point clouds  graphs  time series  images  into forms suitable for   computations of topological summaries  and components dedicated to   extracting sets of scalar features of topological origin  which can be used   alongside other feature extraction methods in scikit learn   Statistical learning with Python                                  Other packages useful for data analysis and machine learning      Pandas  https   pandas pydata org     Tools for working with heterogeneous and   columnar data  relational queries  time series and basic statistics      statsmodels  https   www statsmodels org    Estimating and analysing   statistical models  More focused on statistical tests and less on prediction   than scikit learn      PyMC  https   www pymc io     Bayesian statistical models and   fitting algorithms      Seaborn  https   stanford edu  mwaskom software seaborn     Visualization library based on   matplotlib  It provides a high level interface for drawing attractive statistical graphics      scikit survival  https   scikit survival readthedocs io     A library implementing   models to learn from censored time to event data  also called survival analysis     Models are fully compatible with scikit learn   Recommendation Engine packages                                    implicit  https   github com benfred implicit     Library for implicit   feedback datasets      lightfm  https   github com lyst lightfm    A Python Cython   implementation of a hybrid recommender system      Surprise Lib  https   surpriselib com     Library for explicit feedback   datasets   Domain specific packages                              scikit network  https   scikit network readthedocs io     Machine learning on graphs      scikit image  https   scikit image org     Image processing and computer   vision in python      Natural language toolkit  nltk   https   www nltk org     Natural language   processing and some machine learning      gensim  https   radimrehurek com gensim      A library for topic modelling    document indexing and similarity retrieval     NiLearn  https   nilearn github io     Machine learning for neuro imaging      AstroML  https   www astroml org      Machine learning for astronomy   Translations of scikit learn documentation                                             Translation s purpose is to ease reading and understanding in languages other than English  Its aim is to help people who do not understand English or have doubts about its interpretation  Additionally  some people prefer to read documentation in their native language  but please bear in mind that the only official documentation is the English one   f1     Those translation efforts are community initiatives and we have no control on them  If you want to contribute or report an issue with the translation  please contact the authors of the translation  Some available translations are linked here to improve their dissemination and promote community efforts      Chinese translation  https   sklearn apachecn org         source  https   github com apachecn sklearn doc zh         Persian translation  https   sklearn ir         source  https   github com mehrdad dev scikit learn         Spanish translation  https   qu4nt github io sklearn doc es         source  https   github com qu4nt sklearn doc es         Korean translation  https   panda5176 github io scikit learn korean         source  https   github com panda5176 scikit learn korean           rubric   Footnotes       f1  following  linux documentation Disclaimer     https   www kernel org doc html latest translations index html disclaimer