declearn v2.1.0
Released: 02/03/2023
New features
- Add proper GPU support and device-placement policy utils.
- Add device-placement policy utils:
declearn.utils.DevicePolicy
,declearn.utils.get_policy
anddeclearn.utils.set_policy
. - Implement device-placement support in
TorchModel
,TorchVector
,TensorflowModel
andTensorflowVector
, according to shared API principles (some of which are abstracted intoModel
). - Add tests for these features, and automatic running of unit tests on both CPU and GPU when possible (otherwise, run on CPU only).
- Add device-placement policy utils:
- Add framework-specific
TensorflowOptiModule
andTorchOptiModule
.- Enable wrapping framework-specific optimizer objects into a plug-in
that may be used within a declearn
Optimizer
and jointly with any combination of framework-agnostic plug-ins. - Add functional tests that verify our implementations of the
Adam
,Adagrad
andRMSprop
optimizers are equivalent to these of the Tensorflow and Torch frameworks.
- Enable wrapping framework-specific optimizer objects into a plug-in
that may be used within a declearn
- Add
declearn.metrics.RSquared
metric to compute a regression's R^2. - Fix handling of frozen weights in
TensorflowModel
andTorchModel
.- Add
trainable: bool=False
parameter toModel.get_weights
andModel.set_weights
to enable excluding frozen weights from I/O. - Use
Model.get_weights(trainable=True)
inOptimizer
methods, enabling to use loss-regularizationRegularizer
plug-ins and weight decay with models that have some frozen weights. - Use
Model.set_weights(trainable=True)
and its counterpart to remove some unrequired communications and server-side aggregator and optimizer computations.
- Add
- Fix handling of
tf.IndexedSlices
structures inTensorflowVector
.- Avoid the (mostly silent, depending on tensorflow version) conversion
of
tf.IndexedSlices
row-sparse gradients to a dense tensor every time it can be avoided. - Warn about that conversion when it happens (unless the contexts is known to require it, e.g. as part of noise-addition optimodules).
- Avoid the (mostly silent, depending on tensorflow version) conversion
of
Other changes
- Change
Scaffold.collect_aux_var
behavior on unused optimodule.- Previously, the method would raise an error if
run
had not been called. - Now, a warning is emitted, but a scalar value is returned, that the server- side plugin processed into ignoring that client.
- Previously, the method would raise an error if
- Add SAN capabilities to
declearn.test_utils.generate_ssl_certificates
.- Subject Alternative Names (SAN) enable having an SSL certificate cover the various IPs and/or domain names of a server.
- The declearn interface requires OpenSSL >=3.0 to use the new parameters.
- Add a functional convergence test on a toy regression problem.
- Generate a toy regression problem, that requires regularization.
- Run a scikit-learn baseline and a declearn centralized-case one.
- Run a declearn federated learning pipeline, for all frameworks.
- Verify that in all cases, the model converges to a R^2 >= 0.999.
- Add some assertion utils under
declearn.test_utils
to refactor or enhance some existing and newly-introduced unit and functional tests.
Other fixes
- Fix some newly-identified backend-based issues in
Vector
:- Enable
NumpyVector + <Tensor>Vector
(previously, only<Tensor>Vector + NumpyVector
would work), for all base operations (+ - / *
). - Fix scalar tensors' unpacking from serialized
TensorflowVector
andTorchVector
. - Fix
NumpyVector.sum()
resulting in scalars rather than 0-d numpy arrays. - Improve the documentation of
Vector
and its subclasses.
- Enable
- Fix optimizer plug-ins' framework-equivalence test.
- Fix
pip install declearn
installing network communication third-party dependencies in spite of their being documented (and supported) as optional.- This fix was backported to release
declearn-2.0.2
.
- This fix was backported to release
- Fix the labeling of output gradients in
TensorflowModel
.- This fix was backported to release
declearn-2.0.1
.
- This fix was backported to release