Software
PyExperimenter
The PyExperimenter is a tool for the automatic execution of experiments, e.g. for machine learning (ML), capturing corresponding results in a unified manner in a database. It is designed based on the assumption that an experiment is uniquely defined by certain inputs, i.e., experiment parameters, and a function computing the results of the experiment based on these input parameters. The set of experiments to be executed can be defined through a configuration file listing the domains of each experiment parameter, or manually through code. Based on the set of experiments defined by the user, PyExperimenter creates a table in the database featuring all experiments identified by their input parameter values and additional information such as the execution status. Once this table has been created, PyExperimenter can be run on any machine, including a distributed cluster. Each PyExperimenter instance automatically pulls open experiments from the database, executes the experiment function provided by the user with the corresponding experiment parameters defining the experiment and writes back the results computed by the function. Possible errors arising during the execution are logged in the database. After all experiments are done, the experiment evaluation table can be easily extracted, e.g. averaging over different seeds.
jPL Framework
The jPL framework was originally created by a student project group under the supervision of the Intelligent Systems Group at Paderborn University. The aim of this framework is to provide a generic framework to evaluate different algorithms in the context of preference learning. Learning algorithms solving the following preference learning problems are implemented so far:
- Collaborative Filtering
- Instance Ranking
- Label Ranking
- Multilabel Classification
- Object Ranking
- Ordinal Classification
- Rank Aggregation
duelpy
This is a python package for solving with Preference Based Multi Armed Bandit problems, also known as dueling bandits. Refer to this paper ("Preference-based Online Learning with Dueling Bandits: A Survey" by Viktor Bengs, Róbert Busa-Fekete, Adil El Mesaoudi-Paul, Eyke Hüllermeier) for an overview of the field.
You can compare the implemented algorithms in an experiment by running
python3 -m duelpy.experiments.cli
The experiments are still rather limited. The command-line interface can
currently only run regret-based comparisons in a limited set of configurations.
Pass the --help flag for more information.
See the documentation for more information about the implemented algorithms.
AILibs
AILibs is a collection of Java libraries related to automated decision making. It currently consists of two building blocks. It is also home of the current version of the AutoML-tool ML-Plan.
- JAICore (Java AI Core) is a collection of projects with basic general purpose AI algorithms mainly in the area of logic reasoning, heuristic search, and machine learning
- softwareconfiguration is a collection of projects related to automatically configuring software systems. Here we also maintain the code for our AutoML flagship ML-Plan