Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.pyc
67 changes: 28 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
# MuGtoolAPI
A first attempt at outlining the main structures related to the tools section of the VRE.
# mg-tool-api

## Introduction
This is a first attempt at outlining the main structures related to the
tools section of the VRE. It implements the specifications detailed in the
This library implements the specifications detailed in the
current version (23/09/2016) of the Deliverable 6.1 document: "Design of
computational architecture of software modules" (http://bit.ly/MuGD6_1). It
extends the above document with the aim to provide a simple programming
Expand All @@ -30,31 +28,29 @@ data retrieval; this also makes changes in the DMP transparent to the tools.
straightforward to combine them in workflows, using COMPSs "task" decorator and
the COMPSs runtime as the workflow scheduler.

## Objective

This code is intended as a first draft to fuel discussions among all
involved parties (in particular WP4, WP5 and WP6) about the precise details of
the software architecture of the VRE, and as such won't run as valid Python
code out of the box. Please note that the scientific relevance of this is
limited to providing an example of usage of the class structure proposed.

## Implementation overview

1. Resource:
It is the general container for all data that is exchanged within the
VRE. Subclasses, such as the examples outlined below, should deal with the
details of making the resource available from the various data storage
facilities defined in the DMP. There should be at least one subclass per
storage class in the DMP. Every Resource instance is defined by its data
type, which is immutable. In fact, Resources should probably be immutable
altogether (in the following they are, but aren't forced to be).
2. Tool:
It is the top-level wrapper for tools within the VRE; each tool is defined
by its input and output formats, and by its specific requirements on the
execution environment. Tools should implement a "run" method, that defines
operations needed to get from input to output. Decorating the "run" method
with "@constraints" allows developers to define the execution environment,
while decorating with "@task" makes the tool workflow-ready.
1. Tool:
Is the top-level wrapper for tools within the VRE; each tool is defined
by its input and output formats, and by its specific requirements on the
execution environment. Tools should implement a "run" method, that defines
operations needed to get from input to output. The "run" method calls other
methods which are decorated using PyCOMPSs "@task" and "@constraints",
allowing tool developers to define the workflow and execution environment.
2. App:
Is the main entry point to the tools layer of the VRE; it deals with heterogeneity
in the way Tools are run, in terms of filesystem access, runtime environment,
error reporting, and more. Therefore, Apps are compatible with all Tools.
Apps implement a "launch" method, which prepares and runs a single instance of Tool.
The "apps" module provides some example Apps for straightforward cases:

- *LocalApp*: uses the MuG DMP API to retrieve file names that are assumed
to be locally accessible;

- *PyCOMPSsApp*: specific for Tools using PyCOMPSs;

- *WorkflowApp*: inherits from both of the above, and implements the ability
to unstage intermediate outputs.

3. mug_datatypes:
A repository of all data types available in MuG; it should closely mirror
the contents of the DMP.
Expand All @@ -68,17 +64,10 @@ limited to providing an example of usage of the class structure proposed.
others not related to the Tool should not be caught. The Tool also takes
care of attaching error information to the output resource.

## Examples
See the documentation for the classes for more information.

The "example_docking_workflow.py" contains a minimal implementation of a few
simple example tools in order to perform:
## Examples

a. Ab-initio predict the structure of a protein given its sequence;
b. Ab-initio predict the structure of a DNA oligomer given its sequence;
c. Rigid-body dock two molecules given their 3D structure.
The "examples" module contains usage examples.

These tools wrap external resources, Python libraries, binary executables or
REST APIs, to perform their task; in this example these are not implemented,
but may be replaced with existing tools in the future. Finally, an example
simple workflow is outlined to generate a protein-DNA complex given the
sequence of the two partners using the tools defined above.
The "summer.py" example implements a workflow using PyCOMPSs.
69 changes: 19 additions & 50 deletions __init__.py
Original file line number Diff line number Diff line change
@@ -1,50 +1,19 @@
if __name__ == "__main__":

## Mock the MuG library
class datatypes(object):
Sequence = 0
PDBStructure = 1

class conversion(object):
def mol_to_pdb(molfile, pdbfile):
pass

class mug(object):
datatypes = datatypes
conversion = conversion

import sys
sys.modules["mug"] = mug


## Mock pycompss
class parameter(object):
IN=0
OUT=1
#...

class task(object):
@staticmethod
def task(**kwargs):
return lambda x:x

class constraint(object):
@staticmethod
def constraint(**kwargs):
return lambda x:x

import sys
sys.modules["pycompss"] = object()
sys.modules["pycompss.api"] = object()
sys.modules["pycompss.api.task"] = task
sys.modules["pycompss.api.constraint"] = constraint
sys.modules["pycompss.api.parameter"] = parameter

## Import and execute workflow
sys.path.append("/Users/marco/sci/MuG/src/D6.1")
from MuGtoolAPI import tool, example_docking_workflow, resource

protein_sequence = resource.Resource(mug.datatypes.Sequence)
dna_sequence = resource.Resource(mug.datatypes.Sequence)
example_docking_workflow.DockingWorkflow().run(
protein_sequence, dna_sequence)
"""
Copyright (C) 2016 Marco Pasi <mf.pasi@gmail.com>

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
"""

from app import App
from metadata import Metadata
from tool import Tool
from workflow import Workflow
import apps
import examples

__author__ = 'Marco Pasi, Javier Conejero'
__version__ = '0.3'
__license__ = 'GPL v2'
Loading