declearn.quickrun.quickrun

Run a server and its clients parallelly using asyncio.

The script requires to be provided with the path to a TOML file with all the elements required to configurate an FL experiment, or the path to a folder containing :

• A TOML file with all the elements required to configure an FL experiment.
• A python file in which a declearn model is instantiated (in main scope).
• A data folder, structured in a specific way: folder/ [client_a]/ train_data.(csv|npy|sparse|svmlight) train_target.(csv|npy|sparse|svmlight) valid_data.(csv|npy|sparse|svmlight) valid_target.(csv|npy|sparse|svmlight) [client_b]/ ... ...

Parameters:

Name	Type	Description	Default
config	str	Path to either a toml file or a properly formatted folder containing the elements required to launch the experiment.	required

Notes

• The data folder structure may be obtained by using the declearn-split commandline entry-point, or the declearn.dataset.split_data util.
• The quickrun mode works by simulating a federated learning process, where all clients operate under parallel python processes, and communicate over the localhost using un-encrypted websockets communications.
• When run without any argument, this script/function operates on a basic MNIST example, for demonstration purposes.
• You may refer to a more detailed MNIST example on our GitLab repository. See the examples/mnist_quickrun folder.

Source code in declearn/quickrun/_run.py

async def quickrun(config: str) -> None:
    """Run a server and its clients parallelly using asyncio.

    The script requires to be provided with the path to a TOML file
    with all the elements required to configurate an FL experiment,
    or the path to a folder containing :

    * A TOML file with all the elements required to configure an FL experiment.
    * A python file in which a declearn model is instantiated (in main scope).
    * A data folder, structured in a specific way:
        folder/
            [client_a]/
                train_data.(csv|npy|sparse|svmlight)
                train_target.(csv|npy|sparse|svmlight)
                valid_data.(csv|npy|sparse|svmlight)
                valid_target.(csv|npy|sparse|svmlight)
            [client_b]/
                ...
            ...

    Parameters
    ----------
    config: str
        Path to either a toml file or a properly formatted folder
        containing the elements required to launch the experiment.

    Notes
    -----
    - The data folder structure may be obtained by using the `declearn-split`
      commandline entry-point, or the `declearn.dataset.split_data` util.
    - The quickrun mode works by simulating a federated learning process, where
      all clients operate under parallel python processes, and communicate over
      the localhost using un-encrypted websockets communications.
    - When run without any argument, this script/function operates on a basic
      MNIST example, for demonstration purposes.
    - You may refer to a more detailed MNIST example on our GitLab repository.
      See the `examples/mnist_quickrun` folder.
    """
    toml, folder = get_toml_folder(config)
    # Locate split data or split it if needed.
    client_dict = locate_split_data(toml, folder)
    # Parse toml files.
    ntk_server_cfg = NetworkServerConfig.from_toml(toml, False, "network")
    ntk_client_cfg = server_to_client_network(ntk_server_cfg)
    optim_cgf = FLOptimConfig.from_toml(toml, False, "optim")
    run_cfg = FLRunConfig.from_toml(toml, False, "run")
    model_cfg = ModelConfig.from_toml(toml, False, "model", True)
    expe_cfg = ExperimentConfig.from_toml(toml, False, "experiment", True)
    # Set up the server and client-wise coroutines.
    coro_server = run_server(
        folder, ntk_server_cfg, model_cfg, optim_cgf, run_cfg, expe_cfg
    )
    coro_clients = [
        run_client(folder, ntk_client_cfg, expe_cfg, name, data_dict)
        for name, data_dict in client_dict.items()
    ]
    # Run each and every coroutine in parallel.
    await asyncio.gather(coro_server, *coro_clients)