AiiDA
Automated Interactive Infrastructure and Database for Computational Science


If you use AiiDA please cite:
AiiDA 1.x
Main paper: S.P. Huber et al., Scientific Data 7, 300 (2020)
AiiDA engine: M. Uhrin et al., Comp. Mat. Sci. 187 (2021)

AiiDA 0.x
First paper, ADES model: G. Pizzi et al. Comp. Mat. Sci. 111, 218-230 (2016) (open access version)

Posts

AiiDAlab paper published

👤 🕔 December 16, 2020 Comments Off on AiiDAlab paper published

The paper providing comprehensive information on the architecture and design philosophy of the AiiDAlab web platform was published:

A. V. Yakutovich et al., AiiDAlab – an ecosystem for developing, executing, and sharing scientific workflows, Comp. Mat. Sci. 188, 110165 (2021). DOI:10.1016/j.commatsci.2020.110165

The paper is of interest to anyone wanting to learn more about how AiiDAlab works and how to most effectively use the platform for their own research. A free preprint version of the paper is accessible here: arXiv:2010.02731

All users of AiiDAlab are kindly asked to cite the publication in their own work.

Paper describing AiiDA’s workflow engine published

👤 🕔 November 17, 2020 Comments Off on Paper describing AiiDA’s workflow engine published

The paper describing the details of AiiDA’s workflow engine has been published today:
M. Uhrin et al, Workflows in AiiDA: Engineering a high-throughput, event-based engine for robust and modular computational workflows, Comp. Mat. Sci. 187, 110086 (2021)

In it you will find useful details on how the AiiDA process engine (managing calculation and workflows) works in the AiiDA 1.x series, bringing a huge speed-up with respect to earlier versions also thanks to the event-based design, leveraging the RabbitMQ message broker.

We would be grateful it users running simulations with AiiDA 1.x could cite it along with the main reference paper S.P. Huber et al., Scientific Data 7, 300 (2020).

AiiDA v1.5.0 released

👤 🕔 November 13, 2020 Comments Off on AiiDA v1.5.0 released

This is the fifth minor version of the v1 series and comes with a couple of new features, as well as some relevant bug fixes. Support for Python 3.5 is dropped and support for Python 3.9 is added.

Process functions are now submittable

Since the changes introduced in v1.0, a process function creates a Process instance in the background, just as any other process. This means that, in principle, it could also be serialized and deserialized by a daemon worker from that point onwards. Here we remove the final limitations that prevented process functions from being fully submittable.

It is important to note that the usual rules for making processes submittable apply in this case as well: to be importable by the daemon, it will need to either be registered through an entry point or have its source module available through the PYTHONPATH environment variable. For more information on these requirements, you can check the specific section on launching processes in the AiiDA documentation.

New REST API features

This minor version introduces two new features to the REST API interface of AiiDA. The first one is that the base URL (http://127.0.0.1:5000/api/v4 in a local deployment, for example) will now return a list of the available endpoints. The second one is the addition of a new endpoint full_types_count: this returns the same branched structure that the already existing full_types, but with the addition of a counter key which contains the count of all available node instances of the type indicated.

Archive Refactor

This is part of a series of modifications that aim at making several improvements in the performance and versatility of the process of exporting and importing databases. These changes here should minimally affect the existing interface and might already have some performance boosts, but are an important stepping stone for more significant improvements in the upcoming versions.

For more details on this release, see the change log.

Happy computing!

The AiiDA team

Two 4-year postdoctoral positions (Marzari@EPFL)

👤 🕔 October 26, 2020 Comments Off on Two 4-year postdoctoral positions (Marzari@EPFL)

There are two positions for computational materials scientist/software scientists immediately available in our laboratory for Theory and Simulation of Materials at EPFL. The two positions, for up to 4 years each, are dedicated to data analytics, data integration, and simulation/machine-learning workflows, and are supported by the new H2020 projects DOME4.0 and OpenModel.

The DOME4.0 position is centered around the successful execution of research case studies in combination with the existing Materials Cloud platform (https:/www.materialscloud.org), with a primary focus on the development of machine-learning-based data analytics workflows and auxiliary data services. Successful candidates will develop tools and services required to support their own research aims and those of project partners. Potential scientific projects include the efficient identification of next-generation Li-Ion batteries, machine learning of materials properties from structure and electronic structure, or similar suitable projects.

The OpenModel position is centered around the development of software services to create and execute computational materials science workflows, with a primary focus on the integration of semantic universal data structures and ontology development. Successful candidates would develop tools and services required to support their own research aims and those of project partners. Potential scientific projects include the development and integration of domain ontologies, machine-learning-driven decision-making on models and methods for workflow builders, or similar suitable projects.

Close synergies are envisioned with the Swiss National Centre for Computational Design and Discovery of Novel Materials (NCCR MARVEL), the H2020 MaX Centre of Excellence for Materials Design at the Exascale, the H2020 Materials Modelling MarketPlace, H2020 Intersect and H2020 BIG-MAP Battery2030+, and with the AiiDA and Materials Cloud teams onsite.

Interested candidates should apply by Nov 6, following the instructions at http://theossrv1.epfl.ch/Main/Openings for the two separate openings; the website has also more information on the group. The positions will remain open until filled (the ads will then be removed from the link above). Each position is fully funded for 4 years; the contract is initially for 1 year (as required by EPFL), and renewable yearly upon mutual satisfaction. Gross salaries for a freshly minted Ph.D. start at 83’600 CHF/year (~77’500 EUR/year); social benefits (unemployment, pension) are also provided.

AiiDA v1.4.0 released

👤 🕔 September 24, 2020 Comments Off on AiiDA v1.4.0 released

This is the fourth minor version of the v1 series and comes with some important performance improvements and features, but also various bug fixes. This is the last version to support Python 3.5.

RabbitMQ server can now be hosted on a different machine

In previous versions, AiiDA only supported connecting to a RabbitMQ instance running on localhost. With v1.4, one can now configure AiiDA to connect to a RabbitMQ instance over TCP/IP with optional SSL enabled. Existing profiles will be automatically migrated to add the standard configuration. To see how a new profile can be configured to connect to a remote RabbitMQ instance, please refer to the documentation.

Group extras

Groups now support setting “extras”: just like nodes, you can now store arbitrary metadata on group instances, as long as it is JSON serializable. The interface to setting, getting and deleting extras on a group is exactly the same as that for nodes. Note that, just as they are for nodes, extras are mutable. It is therefore not advised to use them to store information that is important for data provenance.

Scheduler output parsing

This version comes with a new method for the scheduler base class Scheduler.parse_output that can be implemented by scheduler plugins to parse the output that is generated by the scheduler when running a calculation job. The output consists of the detailed job info (if implemented by the scheduler plugin) and the output written to the stdout and stderr file descriptors. This information can hold important information about how the job terminated. It can be used to determine common problems, such as out-of-memory issues or out-of-walltime errors. The parse_output can return a corresponding exit code that is defined on the CalcJob class. You can take a look at the SlurmScheduler.parse_output implementation as an example of how it parses the OOM and OOW errors. The scheduler output parsing, if implemented, will be called by the engine before the output Parser is called, if any. If the scheduler parsing returns an exit code, the engine will already set it on the node but the output parser will still be called. This allows the output parser to still try to parse some data from the retrieved files, and if desired, override the exit code of the scheduler with a more specific exit code. For details on this process and the interaction between exit codes of scheduler and output parsers, see this section of the documentation.

For more details on this release, see the change log.

Happy computing!

The AiiDA team

AiiDA and Materials Cloud papers published in Nature’s Scientific Data

👤 🕔 September 10, 2020 Comments Off on AiiDA and Materials Cloud papers published in Nature’s Scientific Data

The new paper on AiiDA (describing all the work happened in the past 4+ years) and the paper describing the Materials Cloud (an Open Science Platform designed to enable the seamless sharing of resources in computational materials science) have been published on the Nature’s journal Scientific Data.

You can read the full press story on the NCCR MARVEL website, and you can access them (open access) here:

AiiDA virtual tutorial report

👤 🕔 July 27, 2020 Comments Off on AiiDA virtual tutorial report

A few weeks ago we organised the first virtual edition of our AiiDA tutorial week. Arranging a tutorial for 13 different time zones was challenging, but it was a great experience and we received a lot of positive feedback on the virtual format by the participants. You can read the full report here:

http://www.aiida.net/report-on-the-2020-aiida-virtual-tutorial/

In case you missed the tutorial, you can find all the presentations and hands-on material on the webpage of the event:

https://aiida-tutorials.readthedocs.io/en/latest/pages/2020_Intro_Week/index.html

Thanks again to the participants for attending! We look forward to organising more virtual events in the future.

AiiDA v1.3.0 released

👤 🕔 July 21, 2020 Comments Off on AiiDA v1.3.0 released

A new AiiDA release v1.3.0 is available! You can find more information at our download page. It can be installed through pip as:

    pip install aiida-core==1.3.0
    reentry scan

This is the third minor version of the v1 series and comes with some important performance improvements and features, but also various bug fixes. In addition, the documentation has received a major overhaul, both in structure as well as appearance.

Engine performance

One of the big improvements of AiiDA v1.0 was the switch to an event-based engine that makes it a lot more responsive and performant. Running processes broadcast messages, such as when their state changes, that others, such as the daemon workers, can respond to. In this release, this mechanism is used to make the engine even more responsive. A work chain that runs subprocesses, will now start listening to their state changes, such that once the last running subprocess messages that it has finished, the work chain can resume to the next step immediately. This event-based messaging makes the work chains very responsive, however, it is possible that now and again a message slips through the cracks, in which case the work chain would be waiting indefinitely. As a backup, we keep a polling mechanism that will check the status of each subprocess every interval to see whether it has already finished. In previous releases, this was the only mechanism and it would poll every second. This would keep the computer running AiiDA unnecessarily busy. With the new broadcasting mechanism, we were able to increase the polling interval from 1 to 60 seconds. You can even customize this if you would like:

    verdi config runner.poll.interval 120

Note that this polling mechanism is really a backup system and the engine will not rely on it in the vast majority of cases. Still, if you don’t mind waiting a bit longer in this very unlikely case, you can always increase the interval a bit more.

For more details on this release, see the change log.

Happy computing!

The AiiDA team

AiiDA virtual tutorial registration now open

👤 🕔 June 4, 2020 Comments Off on AiiDA virtual tutorial registration now open

The AiiDA team are pleased to announce that registration for our annual tutorial week (July 7-10 2020) is now open until June 19th.

In light of the COVID-19 pandemic, this tutorial week will be hosted virtually. Talks will be pre-recorded and made available to participants before the event, and hands-on tutorials will be held via Zoom at two timeslots, to accommodate for participants in different time-zones.

The goal of this 4 day-tutorial is to help students and researchers from the field of computational materials science get started with writing reproducible workflows. Participation is free of charge and will cater for up to 80 participants.

For further information and the tentative schedule, and to register, please see the official event page.

Documentation revamp, and more than 50 registered plugins

👤 🕔 June 3, 2020 Comments Off on Documentation revamp, and more than 50 registered plugins
We are thrilled to announce a comprehensive revamp of the AiiDA documentation.
 
With the ultimate goal to lower the entry barrier for new users and to make it easier for both beginners and experienced AiiDA users to find the information they are looking for, the documentation is now divided into four distinct sections: a brief, learning-oriented tutorial, goal-oriented how-to recipes, understanding-oriented topic guides, and information-oriented reference material (for more details on the philosophy, see our wiki). Your feedback is most welcome!
 
We are also happy to report that the AiiDA plugin registry  has reached the milestone of 50 plugin packages (and 100 calculation plugins) – all thanks to your continued contribution to the AiiDA ecosystem.