AiiDA
Automated Interactive Infrastructure and Database for Computational Science

AiiDA is a flexible and scalable informatics' infrastructure to manage, preserve, and disseminate the simulations, data, and workflows of modern-day computational science. Able to store the full provenance of each object, and based on a tailored database built for efficient data mining of heterogeneous results, AiiDA gives the user the ability to interact seamlessly with any number of remote HPC resources and codes, thanks to its flexible plugin interface and workflow engine for the automation of complex sequences of simulations.

Journal ref: G. Pizzi, A. Cepellotti, R. Sabatini, N. Marzari, and B. Kozinsky, AiiDA: automated interactive infrastructure and database for computational science, Comp. Mat. Sci. 111, 218-230 (2016)

Open access link: arXiv:1504.0116

Posts

AiiDA project & Google Season of Docs

👤 🕔 April 22, 2019 Comments Off on AiiDA project & Google Season of Docs


The AiiDA project is happy to announce its application to participate in this year’s Google Season of Docs (GSoD). GSoD tries to bring open source projects and technical writers together: To give technical writers experience in contributing to open source projects and help opens source projects to improve their documentation and interact with the technical writing community.

What happens next

See also the full timeline:

  • April 30, 2019 Google publishes the list of accepted mentoring organizations
  • May 29, 2019 Start of technical writer application period. More details about the application process can be found in the technical writer guide, where also the application form will be published
  • June 28, 2019 End of technical writer application period

Prospective technical writers

Are you a technical writer with a background in (or affinity to) computational materials science, physics or chemistry? Would you be interested in boosting a tool that lets researchers manage thousands of simultaneous calculations on supercomputers and helps them work in an open-science environment, making their research reproducible and FAIR? Then please read on!

If you would like to discuss your application or project ideas, please feel free to contact Kevin or any of the other coordinators and prospective mentors:

What is AiiDA?

Simulations and high-throughput computational screenings have emerged as indispensable tools in materials’ science. The important impact of quantum simulations on science in general is underscored by the fact that 12 of the 100 most cited papers of the entire scientific literature deal with simulations based on density functional-theory .

Computational materials science involves screening thousands to hundreds of thousands of materials for key figures of merit. Running thousands of calculations is a nontrivial task in itself and calls for an infrastructure to help orchestrate the calculations and organize the output data, while also ensuring that others can easily reproduce the work. To this aim, there are two important requirements for such an infrastructure: First, it must be flexible enough to support a large variety of simulation workflows; second, it should require minimal overhead compared to the ‘conventional’ way of running simulations.

AiiDA is a python package that is built to make this possible: while automating simulations on supercomputers, it stores the full provenance of data and calculations in a directed graph (see Figure 1c) that can be easily queried using a high-level interface. Moreover, it provides a workflow engine that make it possible to automate and run complex simulations with simple, small snippets of code by using predefined ‘turn-key’ workflows (see Figure 1). Currently, 34 plugins provide support different codes and provide over 50 workflows to compute basic and advanced materials properties.

AiiDA workflows
Figure 1: Complex computational workflow for the Aluminum band structure (a) started using a turnkey workflow as shown in (b). The resulting computational graph (c) that is automatically stored in the AiiDA database. Source: Pizzi, G., Togo, A., & Kozinsky, B. (2018). Provenance, workflows, and crystallographic tools in materials science: AiiDA, spglib, and seekpath. MRS Bulletin, 43(9), 696-702. doi:10.1557/mrs.2018.203

Project ideas

AiiDA is documented on ReadTheDocs and also offers a variety of tutorial material from university classes, schools and workshops (https://aiidateam.github.io/aiida-tutorials/ , https://github.com/aiidateam/aiida_demos, https://github.com/aiidateam/aiida-max-examples, https://aiida-core.readthedocs.io/en/stable/tutorial/index.html) most of which were written for AiiDA 0.x.

Following several alpha and beta versions, 2019 will see the release of AiiDA version 1.0. AiiDA 1.0 brings a number of new features, a redesign of the provenance model as well as of many AiiDA internals. The projects for the 2019 GSoD will focus on updating the available documentation and tutorial material to reflect the changes which were introduced in AiiDA 1.0.

Note: These are project ideas. We highly welcome your suggestions on how to make the documentation more effective as well as how to best accommodate your talents as a technical writer.

AiiDA Tutorials: Teaching AiiDA

Recently, a new major release of AiiDA changed several parts of the API wherefore existing tutorials (https://aiidateam.github.io/aiida-tutorials/ , https://github.com/aiidateam/aiida_demos, https://github.com/aiidateam/aiida-max-examples, https://aiida-core.readthedocs.io/en/stable/tutorial/index.html) need to be updated to reflect these changes.

Moreover, from the perspective of a new user, it would be probably easier to find all existing tutorials in a consistent and updated form in one place (e.g. a with tag with the versions on which the tutorials were tested, see also http://matgenb.materialsvirtuallab.org/). On top of that, several codes that AiIDA supports are not represented with a tutorial — it would be great to add new jupyter notebooks, ideally on MyBinder, showcasing how to use AiiDA with those plugins.
We can suggest plugins which are well developed and contain advanced features (and for which we know the authors) and can help the technical writers in prioritizing.
The technical writer could then test examples from the plugin repositories and create a tutorial, maybe even in form of a screencast (some videos can already be found on the Materials Cloud), explaining a common workflow using AiiDA and one of the many AiiDA plugins.
Existing videos and screencasts could be updated, or at least annotated, to reflect the changes with the new major release. An ambitious technical writer might even set out to develop a online course teaching the basics of AiiDA (writing workflows, querying the database) in the form of a MOOC. An bit less ambitious goal would be to bring existing tutorials on our youtube channel into a consistent format.

AiiDA docs

The AiiDA documentation, written in Sphinx and hosted on ReadTheDocs, is our main effort to document the usage and development of AiiDA. A technical writer could contribute in several ways to this documentation: First, the complete documentation needs to be checked for compatibility with the AiiDA 1.0 release. Second, the documentation can still be extended and refactored to improve the user experience. Sometimes, we encounter issues that are not documented or are not clear enough, or we realise that the organisation of the documentation does not make it easy to find the section that is being searched for. Hence, it would be great to check for frequent issues on the mailing list and include those in the documentation.

AiiDA web site

We feel that our web page could to be updated to better highlight the main features of AiiDA. Currently, the feature section is hidden and should be updated to reflect the feature set of the most recent release. A technical writer could help developing this section and writing a ‘why use AiiDA’ section, ideally supported by appealing visualizations.
We would welcome interactive feature demonstrations or a carousel with examples in which AiiDA was used. This could include links to the databases on the Materials Cloud archive or even interviews with researchers, discussing how they used AiiDA in their research and publication process.

AiiDA plugins migration workshop – EPFL, 25-29 March 2019

👤 🕔 January 11, 2019 Comments Off on AiiDA plugins migration workshop – EPFL, 25-29 March 2019

An AiiDA plugin migration workshop will be held at EPFL Lausanne, Switzerland, aiming at collecting about 20 participants.

The workshop will start on Monday 25th March at 2PM and end on Friday 29th March at 1PM.

We are in the process of preparing the AiiDA code towards the 1.0 release, in which we have recently introduced python 2 + python 3 support (already available in the 1.0 alpha releases). While we strive to always maintain back-compatibility, we have realised that there were a few needed improvements to the API and we felt that the 1.0 release was the right moment to introduce them. (Note: while existing “codebases” will need (often very straightforward) migration-which is
what this workshop is for-existing “databases” will be fully compatible and be automatically migrated when users upgrade to AiiDA 1.0).

The aim of this workshop is to directly support AiiDA plugin developers in the migration of their plugins to support both python 2 and python 3, as well as the changes introduced in AiiDA 1.0.

This workshop is focused on migrating existing plugins (a full list of the plugins supporting almost 60 different codes can be found on the AiiDA plugin registry page). Another tutorial for new developers of plugins and workflows will be held in Lausanne in the week 20-24 May 2019 and will be advertised soon (stay tuned!).

We will also reserve some time for discussions on plugin interfaces and their homogenisation and common APIs, exploiting this unique occasion bringing together many plugin developers.

Main topics

  • explanation of the changes on 1.0;
  • hands-on workshop on porting plugins to py2+3 and new aiida 1.0;
  • discussions on common interfaces to different plugins for common functionalities (e.g. crystal structure relaxation, band structure, …);
  • discussion of automated plugin testing against different versions of AiiDA, python, …

Program
Every day there will be an informal coffee break from 11:00 – 11:15 and 16:00 – 16:15

Monday 25th
2:00 – 2:15: Registration
2:15 – 2:20: Introductory talk
2:20 – 3:20: What’s new in v1.0.0 + backwards incompatible changes (Sebastiaan)
3:20 – 4:00: Talk on python2/python3 compatibility
4:00 – 4:30: Coffee break
4:30 – 4:45: Plugin migration – step by step instructions
4:45 – 6:00: Hands-on plugin migration workshop

Tuesday 26th
09:00 – 09:45: Tutorial on plugin versioning and git-flow model of releasing
09:45 – 11:00: Hands-on plugin migration workshop
13:00 – 14:00: Lunch
14:00 – 18:00: Hands-on plugin migration workshop

Wednesday 27th
09:00 – 13:00: Hands-on plugin migration workshop
13:00 – 14:00: Lunch
14:00 – 14:45: Talk on new workflow and engine features
14:45 – 18:00: Hands-on plugin migration workshop
19:00 – 22:00: Social dinner (place to be announced)

Thursday 28th
09:00 – 13:00: Open discussion on plugin common guidelines and common workflow interfaces
13:00 – 14:00: Lunch
14:00 – 18:00: Hands-on plugin migration workshop

Friday 29th
09:00 – 13:00: Hands-on plugin migration workshop

Support

This workshop is supported by the NCCR MARVEL funded by the Swiss National Science Foundation and by the European H2020 MaX Centre of Excellence.