Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The Open Cloud Mesh project is coordinated by GÉANT with contributions from CERN, National Research and Education Networks (NRENs), academic institutions and commercial partners.

Interconnected Private Clouds for Universities and Researchers

Open Cloud Mesh (OCM) is a joint international initiative under the umbrella of the GÉANT Association that is built on the open Federated Cloud Sharing application programming interface (API) - first initiated and implemented by ownCloud Inc. - taking Universal File Access beyond the borders of individual clouds and into a globally interconnected mesh of research clouds without sacrificing any of the advantages in privacy, control and security an on-premises cloud provides. OCM defines a vendor-neutral, common file access layer across an organization and/or across globally interconnected organizations, regardless of the user data locations and choice of clouds.

Concept document

The Open Cloud Mesh concept document was produced by Christian Schmitz at ownCloud Inc. and first distributed on 23 July 2015. The project has been established under the auspices of the GÉANT Association in order to ensure the vendor neutral design and development of the open protocol.

Download from here...

Statement

All leading partners of the Open Cloud Mesh project are fully committed to the open API design principle. This means that - from day one - the OCM sharing API should be discussed, designed and developed as a vendor neutral protocol to be adopted by any on-premise sync&share product vendor or service provider. We acknowledge the fact that the piloting of the first working interface prototype will be carried out in an ownCloud environment that should not effect the adoption of the open API in any other vendor and provider domain.

Community effort - open for participation

A collaborative project was established under the umbrella of GÉANT called the Open Cloud Mesh project on 22 October 2015. The kick-off meeting was held in Vienna, Austria.

The project is co-managed by Peter Szegedi (GÉANT), Jakub Moscicki (CERN) and Christian Schmitz (ownCloud). This combination of project management ensures that all the major stakeholders – GÉANT National Research and Education Networks, CERN research community and ownCloud Inc. as a commercial company with its open source developers community – are equally represented in the project and the technical, management and business aspects are well-covered.

The collaborative project is fully open to any participation and in-kind contributions. Interested parties can subscribe to the mailing list at:

ocm-all@lists.geant.org...

GÉANT Project code P15_225

Key stakeholders

Repository

Key stakeholders

Name

Organization

Interest / Involvement / Role

RACI

Stakeholder Comments

Organizations from the research and/or education space Peter SzegediGÉANTProject managementA, R

Name

Organization

Interest / Involvement / Role

RACI

Stakeholder Comments

Organizations from the research and/or education space
Peter SzegediGÉANTProject managementA, R
Committed

Jakub Moscicki

Massimo Lamanna

CERNProject managementA, RCommitted
Ron TrompertSURFsaraContribute to the specifications and developmentR, CCommitted

Benedikt Wegmann

Ralph Krimmel

GWDGContribute to the specifications and developmentR, CCommitted
Woojin SeokKISTIInterest from South KoreaI 
Universities and Higher Educational Institutions

Holger Angenent

Sciebo / Uni Münster

Contribute to the specifications and development

R, C

Committed

Andreas EckeyTechnische Universität BerlinContribute to the specifications and developmentR, C 
Christian KracherUniversity of ViennaContribute to the specifications and developmentR, CCommitted
National Research and Education Networks
Rogier SpoorSURFnetContribute to the specifications and developmentR, C 

Christoph Herzog

Simon Leinen

SWITCH

Contribute to the specifications and development

R, C

Committed

Guido Aben

David Jericho

AARNet

Contribute to the specifications and development

R, C

Committed

David Antoš

CESNET

Contribute to the specifications and development

R, C

Committed

Frederik Orellana

DeIC

Contribute to the specifications and development

R, C

 

Kurt Bauer

ACOnet

Contribute to the specifications and development

R, C

 

Jari Miettinen

CSC / Funet / EUDAT

Contribute to the specifications and development

R, C

 

Hrachya Astsatryan

Andranik Hayrapetyan

Wahi

ASNET-AMContribute to the specifications and development Committed
Commercial vendors
Christian SchmitzownCloudProject managementA, RCommitted

Charles du Jeu

David Gillard

PydioContribute to the specifications and developmentR, CCommitted
Christian SprajcPowerFolder
InterestedI 
Implementing the latest specs in their product.R, C Committed
Russell AlbertZettaboxInterestedI 
Frank KarlitschekNextcloudContribute to the specifications and developmentR, CCommitted
Vlad RomanFileRunInterestedI 
Johnatan XuSeaFileInterestedI

The project will be delivered in phases.

Project plan (Phase III.)

TBC...

June 2016 - January 2017 


OCM - Phase

II

IV.

Phase IIIV. aims at demonstrating the OCM protocol first implemented and working between two independent sync&share software vendors' domains. A live demonstration happened at TNC'16 in June 2016.

February 2016 - June 2016

paving the way towards standardization. Explore patent and IPR issues, as well as the potential fora for initiating the standardization discussion. Need a reference installation (proxy) fully complaint with the latest specs.

October 2017 - May 2018

PHASE IV
PHASE II
.DESCRIPTIONCOMMENTASSIGNED TOSTART DATEEND DATESTATUS
1. Pre-project (preparation)

Initiation

Pick up the results on Phase I.

See section 4.2 Deliverable of Phase I. below...

Create the new structure of two WGs:

  1. Strategic WG ocm-admin@lists.geant.org
  2. Technical WG ocm-tech@lists.geant.org

Assign mailing lists above

All: ocm-all@lists.geant.org

Peter Szegedi19 January 20165 February 2016 StatuscolourGreentitleCOMPLETE2. Initiation      2.1Call for a kick-off video conference
What are the next steps for OCM?
Understanding current usage of Federated Sharing feature
- How many ownCloud sites in our community have this feature enabled? 
- How many admins know about this feature? 
- What prevents them from enabling it? 
- If enabled, how many users have already used this functionality.
[ACTION] Christian to send a questionnaire to all known ownCloud instance admins.
  • [ACTION] Holger to prepare simple admin instructions on how to check usage and status.
  • SIG-CICC discussion in Amsterdam

    Action 3 - Build a reference proxy/gateway implementation of the agreed OpenCloudMesh (OCM) federated sharing protocol specification to support the on-boarding of closed-source EFSS solutions as well as the compliance of the current open-source products.

    Peter to present the idea to the GCC. Secure funding! Idea was presented, no conclusions were drawn.

    Next GCC meeting: 22 February 2018

    Peter Szegedi, Guido Aben, Jakub Moscicki, Maciej Brzezniak26 September, 201724 October 2017

    Status
    colourYellow
    titleon going

    2. Initiation





    2.1Secure funding and developers
    • GCC small project's budget
    • OCM own budget
    • Partners' contributions
    • Vietsch Foundation project
    Peter
    24 October 2017

    Status
    colourYellow
    titleon going

    2.2Create the development teams and coordinateSynergies...
    • CERN - developing OCM for CERNBox (case study for Up2U)
    • PSNC - developing OCM for SeaFile (closed source - proxy approach)
    • PowerFolder is developing OCM 0.0.3 for their product....

    Discussion at the CS3 workshop in Krakow 29 - 31 January 2018

    http://cs3.cyfronet.pl/

    ownCloud and Nextcloud agreed to revisit their current OCM protocol implementations and work out the next iteration of the protocol implementation in their respective products. This agreement was welcomed by all the CS3 participants and the OCM project partners. PowerFolder and Pydio agreed to review the implementations done by ownCloud/Nextcloud and adjust their specs. SeaFile expressed their interest in implementing the protocol in their product too.

    There seems to be a need for a neutral testing/sandboxing environment where the various vendor products can be tested against one an other in terms of interoperability and specs compliance. GÉANT was asked to investigate the possibility to host/manage/operate such a sandbox environment.

     

    Kuba, Maciej, Peter, Christian
    31 January 2018
    General feeling is that we do not have enough understanding of this technology
    Tilo proposed to setup a test bed where we could check various technical aspects (ref: http://cs3.ethz.ch/presentations/Interoperability/02Moscicki.pdf) and get confidence.
    • [ACTION] Kuba to send preliminary test plan.
    Involvement of pioneer users
    Tilo proposed to involve pioneer users early on to get real user feedback. The extent of exposure of the users should be function of progress in point (2). 
    • [ACTION] All participants of the call are asked to look for potential users that would be willing to try out this functionality. There should be a real use-case for it e.g. members of the same research group sitting in different locations and using private clouds in their institutes already.
    Possible candidate users identified already: 
      - Physicists from ETH and CERN.
      - Others...
    Christian Schmitz19 January 201611 February 2016

    Status
    colourGreen
    titleCOMPLETE

    2.2Define objectives, key results and timelines
    • Repeat the Zurich demo but with more domains and more interesting use cases.
    • Demonstrate the interest and if possible the active participation of vendors other than ownCloud Inc.
    Peter Szegedi11 February 201611 March 2016 StatuscolourGreentitleCOMPLETE3.  Subsequent stages (execution)      3.1Build the OCM community

    ownCloud v.8.2 or higher: Uni Münster, Uni Vienna, AARnet, SWITCH, CESNET, GWDG

    Others very close: SURFnet, CERN

    Pydio v.6.4: ASNET-AM, University of Lausanne (TBC)

    Peter Szegedi19 January 2016  StatuscolourYellowtitleon going3.2KEY - Involvement of other vendors...

    3.  Subsequent stages (execution)
    TO BE DEFINED...



    3.1





    3.2





    3.3





    4. Delivery





    4.1





    4.2





    5. Closing






    OCM - Phase III.

    Phase III. aims at creating a protocol description/definition that is compliant, described, neutral, modular, minimal, secure and robust in order to be implemented by any vendors.

    June 2016 - January 2017

    PHASE III.DESCRIPTIONCOMMENTASSIGNED TOSTART DATEEND DATESTATUS
    1. Pre-project (preparation)

    Initiation

    Pick up the results on Phase II.

    See Section 5. of Phase II. below...

    • Find a professional protocol designer who will describe OCM in a more formalized way (using Swagger Framework or similar). 
    • Establish a reference infrastructure and test environment where the implementations can be validated against.

    Meeting with Apiwise on 12/07/2016

    Peter Szegedi30 June 20161 August 2016

    Status
    colourGreen
    titleCOMPLETE

    2. Initiation      
    2.1.Collecting voluntary contributions

    The OCM partners have been approached to source the vendor-agnostic, design-first formalization of the OCM API using standard frameworks and proper documentation. 

    The following 8 partners offered voluntary contributions in the range of EUR 1,000 - 2,500 installments:

    AARNet, Nextcloud, Sciebo, ownCloud, Uni Vienna, CESNET, GWDG and CERN

    We have collected sufficient amount of money to contract with the third-party who will deliver the results. 

    VERY MUCH APPRECIATED TO ALL CONTRIBUTING PARTNERS!!!

    Peter Szegedi1 August 201619 August 2016

    Status
    colourGreen
    titleCOMPLETE

    2.2Administrativa

    All contributing partners need to be invoiced. An MoU and Order Form have been developed to be signed, if needed by the financial offices.

     

     Peter Szegedi 1 August 2016Checking one...

    Status
    colourGreen
    titleCOMPLETE

    2.3Money received by GÉANT

    All contributions received except one. Solution will be put in place by January 2017.

    All DONE. Very much appreciated!

     Peter Szegedi19 August 20161 February 2017

    Status
    colourGreen
    titleCOMPLETE

    3.  Subsequent stages (execution)      
    3.1Offer from third-party API expert

    The Dutch company called APIWISE made an offer for the work that has been accepted by the contributing partners.

    An amendment have been added to define how GÉANT will evaluate the results and what our priorities are.

    Contract is finalized and to be signed by GÉANT CEO.

    Signed!

    Dimitri van Hees - Apiwise

    Peter Szegedi

    19 August 201628 August 2016

    Status
    colourGreen
    titleCOMPLETE

    3.2Tech meeting I.18 November 2016

    The former OCM specifications are now completely (i.e. all modules) translated to https://docs.apiwise.nl/ocm/index.html

    There was a discussion about priorities, what should be covered by the new specs and what not. The functional modules are:

    1. sharing and federated sharing (this is the CORE)
    2. synchronisation and accessing the file using WebDAV or other file transfer protocol (this could be only a starting point for discussion)
    3. service/user discovery (next thing)
    4. what to do with the files/folders after sharing... (outside of scope)

    Third party view of APIWise would be appreciated on the overall architecture.

    A lot of historical decisions had been made that can be revised by APIWise. The sharing part doesn't have to comply with the old Open Collaboration Services specification.

    How security works: This must be included in the core stuff. How to trust the other services. Grant and revoke privileges to access the API by other partners. This will be rolled out in next OC/NC versions.

    Some vendors don't use WebDAV. Pydio doesn't like WebDAV that is old schools, however supports it. Use it because all the other solutions are proprietary but think about new ways.

    Based on HTTP and stateless (rest) protocols is good if only domains are talking but if others (e.g. mobile) want to build apps on top of the API that's a different story.

    APIWise can suggest on the new file transfer protocol part as a separate module exchanging WebDAV.

    The guiding rule for now should be to make it easy as possible to be implemented by other sync&share vendors.

    Lookup is an other big topic. This could be the focus after file transfer. Exchanging address books and listing remote servers will be implemented in OC/NC 9.0.

    We have to make sure that the API is versioned!

    Stick to WebDAV for now, but there are several interpretations of the specific parts. It is not just so easy to rely on WebDAV server and expect it to work. Special attributes ITags fileIDs, etc. are optionally added in OC. Pydio doesn't have them.

    Don't try perfect in one step. Staging, modules, versioning.

    Document the interaction between remote servers. It is not documented yet. Pydio had identified specification holes. Need to be fixed.

    SUMMARY

    1. Core speccs: federating sharing and sharing API. Modernize it in a vendor neutral way. Don't stick to legacy. This must be completed first. Must be done by the CS3 workshop (preferably mid December). APIwise has some ideas on how to modernize the federated sharing module to be shared in a written document.

    2. Think about file transferring. Won't be completed but start thinking about it. There's less freedom here. Start with WebDAV and the missing documentation. Looking into extensions that OC implemented that are optional to others. By the CS3 Workshop we should at least have a good understanding.

    Next meeting in 3-4 weeks.

    All18 November 201618 November 2016

    Status
    colourGreen
    titleCOMPLETE

    3.3Tech meeting II.

    16 December 2016

    1) Open issues and clarifications

    We discussed 3 open issues where we needed agreements in order to finalize the work. The full list of issues are at
    https://github.com/GEANT/OCM-API/issues

    - Issue #24 and #25 Sharing proposal: federated sharing w/o acceptance from consumer

    We agreed that notification about the acceptance of a share (before access) SHOULD be done, but it's not mandatory by the protocol specifications. It is not functional part of the access flow but SHOULD be implemented as a separate "friendly" notification flow at the application layer.

    - Issue #23 Security proposal: trusted services and shared secrets / request signing

    We agreed that creating a trusted circle of services is recommended but not functional part of the protocol. The Authentication of the service that offers a share MUST be part of the protocol but the Authorization mechanism is out of scope. The implementation of a potential "whitelisting" authorization mechanism is up to the actual service owner.

    - Issue #26 How does a provider know where to send the invitation to?

    We agreed that service domain of a particular users is an a-priori knowledge. It is up to the actual implementation whether the user offering the share must specify the service domain explicitly or it can be automatically looked up e.g, via an IdP as an attribute of the user.

    Organise a call with the GÉANT AAI experts to understand what user attributes can be requested from the IdPs via federations and eduGAIN.

    - Discussion about the actual share.

    We agreed that currently the share is a WebDAV target (URL of the resource, plus access token, permissions and other metadata). Agreed to include an attribute that can only be WebDAV today but allows the possibility to negotiate on other protocols in the future.

    2) Roadmap

    - Results considered to be final will be distributed by APIWISE on 16 January 2017.

    - Partners can start reviewing, commenting and discussing via email/ticket and at the CS3 Workshop on 1 February 2017.

    - Based on the comments, if any, results must be finalized and made available by APIWISE on 10 February 2017.

    - Contract closed.

    3) CS3 Workshop

    The OCM session is scheduled on 1 February 2017
    between 11.00 - 13.00 CET. Will be chaired by Peter Szegedi, GÉANT. This is to present the results of the OCM work to the broader audience including vendors.

    4) Next call

    If needed, we can organise a call on the week of 16 January TBC.

    All16 December 201616 December 2016

    Status
    colourGreen
    titleCOMPLETE

    4. Delivery      
    4.1Delivery of the first results

    GitHUB

    https://github.com/GEANT/OCM-API

    API reference documentation

    https://rawgit.com/GEANT/OCM-API/v1/docs.html

    Dimitri van Hees 26 January 2017

    Status
    colourGreen
    titleCOMPLETE

    4.2Review by the partners

    CS3 Workshop hosted by SURFsara in Amsterdam

    https://cs3.surfsara.nl/

    Results have been delivered

    Image Added

    Presentations:

    Peter Szegedi, GÉANT

    https://indico.cern.ch/event/565381/contributions/2401967/attachments/1405158/2146476/OCM-CS3_2017.pdf

    Dimitri van Hees, APIWSE

    https://indico.cern.ch/event/565381/contributions/2401966/attachments/1405101/2146341/ocm.pdf

    Notes:

    Apiwise guys did a great job. They delivered exactly what we asked for.

    Saying that it became clear, especially during the OCM discussion at CS3, that this API spec is only the first step. So there is more work needed to evolve this spec. But this is independent from this current contract with Apiwise.

    Most importantly, the current API implementations at ownCloud, NextCloud and Pidyo do not fully comply with the new API documentation!!!

    Need some development time to converge... before we go forward.

     

    All 1 February 2017

    Status
    colourGreen
    titleCOMPLETE

    5. ClosingAcceptance, next steps...

    Final results to be submitted by APIWISE.

    Close of contract. PAID!

    Dimitri van Hees 10 February 2017

    Status
    colourGreen
    titleCOMPLETE


    OCM - Phase II.

    Phase II. aims at demonstrating the OCM protocol first implemented and working between two independent sync&share software vendors' domains. A live demonstration happened at TNC'16 in June 2016.

    February 2016 - June 2016

    PHASE II.DESCRIPTIONCOMMENTASSIGNED TOSTART DATEEND DATESTATUS
    1. Pre-project (preparation)

    Initiation

    Pick up the results on Phase I.

    See section 4.2 Deliverable of Phase I. below...

    Create the new structure of two WGs:

    1. Strategic WG ocm-admin@lists.geant.org
    2. Technical WG ocm-tech@lists.geant.org

    Assign mailing lists above

    All: ocm-all@lists.geant.org

    Peter Szegedi19 January 20165 February 2016

    Status
    colourGreen
    titleCOMPLETE

    2. Initiation      
    2.1Call for a kick-off video conference
    What are the next steps for OCM?
    Understanding current usage of Federated Sharing feature
    - How many ownCloud sites in our community have this feature enabled? 
    - How many admins know about this feature? 
    - What prevents them from enabling it? 
    - If enabled, how many users have already used this functionality.
    • [ACTION] Christian to send a questionnaire to all known ownCloud instance admins.

    • [ACTION] Holger to prepare simple admin instructions on how to check usage and status.
    General feeling is that we do not have enough understanding of this technology
    Tilo proposed to setup a test bed where we could check various technical aspects (ref: http://cs3.ethz.ch/presentations/Interoperability/02Moscicki.pdf) and get confidence.
    • [ACTION] Kuba to send preliminary test plan.
    Involvement of pioneer users
    Tilo proposed to involve pioneer users early on to get real user feedback. The extent of exposure of the users should be function of progress in point (2). 
    • [ACTION] All participants of the call are asked to look for potential users that would be willing to try out this functionality. There should be a real use-case for it e.g. members of the same research group sitting in different locations and using private clouds in their institutes already.
    Possible candidate users identified already: 
      - Physicists from ETH and CERN.
      - Others...


    Christian Schmitz19 January 201611 February 2016

    Status
    colourGreen
    titleCOMPLETE

    2.2Define objectives, key results and timelines
    • Repeat the Zurich demo but with more domains and more interesting use cases.
    • Demonstrate the interest and if possible the active participation of vendors other than ownCloud Inc.
    Peter Szegedi11 February 201611 March 2016

    Status
    colourGreen
    titleCOMPLETE

    3.  Subsequent stages (execution)      
    3.1Build the OCM community

    ownCloud v.8.2 or higher: Uni Münster, Uni Vienna, AARnet, SWITCH, CESNET, GWDG

    Others very close: SURFnet, DVLA, CERN

    Pydio v.6.4: ASNET-AM, University of Lausanne (TBC)

    Peter Szegedi19 January 2016 

    Status
    colourYellow
    titleon going

    3.2KEY - Involvement of other vendors...

    Locate other vendors and identify user cases, universities with two products and cross-sharing needs.

    18-02-2015

    News from Christian:

    Pydio will most likely join OCM. They are actually aiming for test implementation by end of March.

    16-03-2016

    Charles du Jeu and David Gillard from Pydio (https://pydio.com/) joined the OCM project.

    Charles, the CEO/CTO of Pydio, confirmed that that are preparing the release of an important dev version, that actually contains the Federated Sharing API implementation. See https://pydio.com/en/community/releases/pydio-core/pydio-core-631-development-release for more info. Pydio is making its best to have this release transformed to a stable one by the end of this month.

    Press release

    https://pydio.com/en/blog/federated-sharing-pydio-connects-owncloud-and-joins-opencloudmesh-initiative

    16-05-2016

    Russell Albert from Zettabox signaled his interest in OCM. Discussion is on-going...

    10-06-2016

    Christian Sprajc from PowerFolder signaled his interest in the OCM project and reported this

    PowerFolder Federated Clouds feature now avnow availableailable: Testers needed!

    We just finished the first version of PowerFolder including federated cloud sharing: https://www.powerfolder.com/powerfolder-11-eap-join-federation/
    PowerFolder in willing to open up their API and comply with the OCM protocol at a later phase.

    15-06-2016

    Frank Karlitschek from Nextcloud (ownCloud fork) joined the OCM project.

     

    Christian Schmitz

    Peter Szegedi

    19 January 201616 March 2016

    Status
    colourGreen
    titleCOMPLETE

    3.3Demonstrate inter-vendor OCM functionality

    29-03-2016 OCM call at 2pm CET

    1) Charles (Pydio) demonstrated via screen-share the OCM functionality that has been implemented by Pydio in their software. Federated sharing of files was shown between two local independent instances of Pydio v.6.4.0.(to be released on 30 March) running locally.

    Comments:

    • Server name and user name must be defined separately. Should work with username@domain@servername (TBC)
    • Shared files are kept in a tree separate from the local file tree (this topic is also being discussed by the OCM community)

    2) Charles (Pydio) and Frank (ownCloud) also tested the interoperability (technical feasibility) of the OCM protocol implementations between Pydio v.6.4 and ownCloud v.9.0 offline. It was reported successful but yet to be seen...

    Screen Shot 2016-04-07 at 14.20.37.pngImage Added

     

    Charles du Jeu

    David Gillard

    Frank and Lukas

    16 March 201629 March 2016

    Status
    colourGreen
    titlecomplete

    3.4

    Multi-vendor OCM validation by the Community

    ownCloud - Pydio

    1) AARNet (Guido) initiated some initial testing with ASNET-AM (Hrachya)

    AARNet (Australia) uses ownCloud and ASNET-AM (Armenia) uses Pydio.

    • Some initial issues out of the box. Pydio developers are involved...
    • Results to be seen...

     

    2) More Pydio users and test scenarios to be defined (see 3.1)

    • RENATER and University of Lausanne to be contacted...

    Guido Aben

    Hrachya Astsatryan

    Andranik Hayrapetyan

    Wahi

      

    Status
    colourGreen
    titlecomplete

    4. Delivery      
    4.1Demonstration at the TNC'16 Conference in Prague, Czech Republic

    DEMONSTRATION ownCloud, Pydio: Interoperability demo at GÉANT booth

    Image Added

    Lightning talk by Guido Aben: submitted and approved

    Presented by Christian Schmitz, ownCloud.

    Christian Schmitz

    Guido Aben

    Charles du Jeu

    19 January 201613 June

    Locate other vendors and identify user cases, universities with two products and cross-sharing needs.

    18-02-2015

    News from Christian:

    Pydio will most likely join OCM. They are actually aiming for test implementation by end of March.

    16-03-2016

    Charles du Jeu and David Gillard from Pydio (https://pydio.com/) joined the OCM project.

    Charles, the CEO/CTO of Pydio, confirmed that that are preparing the release of an important dev version, that actually contains the Federated Sharing API implementation. See https://pydio.com/en/community/releases/pydio-core/pydio-core-631-development-release for more info. Pydio is making its best to have this release transformed to a stable one by the end of this month.

    Press release

    https://pydio.com/en/blog/federated-sharing-pydio-connects-owncloud-and-joins-opencloudmesh-initiative

    16-05-2016

    Russell Albert from Zettabox signaled his interest in OCM. Discussion is on-going...

    10-06-2016

    Christian Sprajc from PowerFolder signaled his interest in the OCM project and reported this

    PowerFolder Federated Clouds feature now available: Testers needed!

    We just finished the first version of PowerFolder including federated cloud sharing: https://www.powerfolder.com/powerfolder-11-eap-join-federation/
    PowerFolder in willing to open up their API and comply with the OCM protocol at a later phase.

    15-06-2016

    Frank Karlitschek from Nextcloud (ownCloud fork) joined the OCM project.

     

    Christian Schmitz

    Peter Szegedi

    19 January 201616 March 2016 StatuscolourGreentitleCOMPLETE3.3Demonstrate inter-vendor OCM functionality

    29-03-2016 OCM call at 2pm CET

    1) Charles (Pydio) demonstrated via screen-share the OCM functionality that has been implemented by Pydio in their software. Federated sharing of files was shown between two local independent instances of Pydio v.6.4.0.(to be released on 30 March) running locally.

    Comments:

    • Server name and user name must be defined separately. Should work with username@domain@servername (TBC)
    • Shared files are kept in a tree separate from the local file tree (this topic is also being discussed by the OCM community)

    2) Charles (Pydio) and Frank (ownCloud) also tested the interoperability (technical feasibility) of the OCM protocol implementations between Pydio v.6.4 and ownCloud v.9.0 offline. It was reported successful but yet to be seen...

    Screen Shot 2016-04-07 at 14.20.37.pngImage Removed

     

    Charles du Jeu

    David Gillard

    Frank and Lukas

    16 March 201629 March
    2016

    Status
    colourGreen
    title

    complete

    Complete

    3.4

    Multi-vendor OCM validation by the Community

    ownCloud - Pydio

    1) AARNet (Guido) initiated some initial testing with ASNET-AM (Hrachya)

    AARNet (Australia) uses ownCloud and ASNET-AM (Armenia) uses Pydio.

    • Some initial issues out of the box. Pydio developers are involved...
    • Results to be seen...

     

    2) More Pydio users and test scenarios to be defined (see 3.1)

    • RENATER and University of Lausanne to be contacted...

    Guido Aben

    Hrachya Astsatryan

    Andranik Hayrapetyan

    Wahi

      
    4.2Demo feedback

    This vision is that the OCM spec should be:

    • compliant: with standard practices of the http world (error codes, conventions,…)
    • described: using industry-strength documentation/testing system (e.g. swagger.io)
    • neutral: should not have any artifacts or assumptions stemming directly from particular implementation or implementation language
    • modular: allow providers to implement minimal functionality and add optional components of the spec as they please (or not)
    • minimal: offload as much as possible of additional functionality to existing mechanisms in the network, especially for optional modules (e.g. lookup)
    • secure: compliant with modern security frameworks (e.g. OAuth2, JWT, …) For the modules, I would consider at first: - auth/autz negotiation - sharing of files - synchronization of files - user discovery (optional)
    • robust: implementations should continue to deliver their  service even when interacting with a failed implementation/service or malicious intended attempts at federation as attack vector

    PREFERRED MODULES:

    • auth/autz negotiation
    • sharing of files
    • synchronization of files
    • user discovery (optional)
      30
    StatuscolourGreentitlecomplete4. Delivery      4.1Demonstration at the TNC'16 Conference in Prague, Czech Republic

    DEMONSTRATION ownCloud, Pydio: Interoperability demo at GÉANT booth

    Image Removed

    Lightning talk by Guido Aben: submitted and approved

    Presented by Christian Schmitz, ownCloud.

    Christian Schmitz

    Guido Aben

    Charles du Jeu

    19 January 201613
    June 2016

    Status
    colourGreen
    titleComplete

    4.2Demo feedback

    Collect and made available...

     

    DEMO IDEAS for the next step...

    We can redo the same demo as shown at CS3 but fancier and bigger, also involving more sites. Additional sites which would like to join this time for the demo:
    • SURFNet: currently stuck with the Shibboleth issue with ownCloud 8
    • CERN: needs ownCloud 9 fully deployed and operational (sharing module is not in maintainable/fixable state as of ownCloud 8)
    • Others...
      30 June 2016 StatuscolourYellowtitleon-going5. ClosingPublish results and define next phase

    Latest OCM protocol implementation on GitHub: https://github.com/cernbox/OpenCloudMeshSpecification

    5. ClosingPublish results and define next phase

    Latest OCM protocol implementation on GitHub: https://github.com/cernbox/OpenCloudMeshSpecification

    News item about the OCM demo at TNC'16:

    http://www.geant.org/News_and_Events/Pages/Worlds-first-multi-vendor-server-to-server-cloud-sharing-interoperability-demonstration-powered-by-OpenCloudMesh-.aspx

    Next steps:

    • We'll find a professional protocol designer who will describe OCM in a more formalized way (using Swagger Framework or similar). 
    • We'll establish a reference infrastructure and test environment where the implementations can be validated against. 
      30 June 2016

    Status
    colour

    Yellow

    Green
    title

    on-going

    complete

    OCM - Phase I.

    Phase I. aims at demonstrating the first working prototype of the OCM protocol (API v1.0 BETA) functionally working between two separate administrative ownCloud domains (i.e. between two NRENs).

    October 2015 - February 2016

    PHASE I.DESCRIPTIONCOMMENTASSIGNED TOSTART DATEEND DATESTATUS
    1. Pre-project (preparation)

    Start collecting organizations and people interested in joining the initiative.

    Mailing list to be created. Announcements to be made.

    Peter Szegedi

    Christian Schmitz

    8 February 201515 June 2015
    Status
    colourGreen
    titleCOMPLETE
    2. Initiation      
       2.1

    ownCloud to release the first version of the API code and documentation.

    DELAYED

    Code v.0.002 has been released on 27 July 2015 by ownCloud Inc.

    Comments have been provided by CERN.

    Christian Schmitz8 February 201527 July 2015

    Status
    colourGreen
    titleCOMPLETE

       2.2Create a project team, estimate budget and organize a kick-off meeting.

    VC for coordination on 24 August 2015.

    • Concluded in the Communique GSec(15)015.

    Pre-launch meeting organized by ownCloud on 28 August 2015 in Berlin. Peter (GÉANT), Guido (AARnet) and Kuba (CERN) and others.

    • Code v.0.002 released and commented
    • Christian as an interim leader
    • GÉANT to provide the project framework
      • Find the neutral project lead, co-chairs from the community.
      • Approach other vendors: PowerFolder, W3C, Pydio, Cozy, fixithere
      • Put it in the GÉANT procurement requirements (the support for the API)

    NIF/PID to be submitted and mailing list migration to be done.

    Kick-off meeting: 22 October, 2015 in Vienna, Austria

    The slides of the event can be found here.
    password edu221015

    Peter Szegedi

    Christian Schmitz

    8 February 201522 October 2015

    Status
    colourGreen
    titleCOMPLETE

     

     

    3.  Subsequent stages (execution)      
    3.1Get the API v.0.004 code and documentation, define the participating domains, initiate the first tests.

    DRAFT protocol definition v.0.004 released

    OpenCloudMesh = ownCloudMesh

    Christian Schmitz15 June 201528 Augustus 2015

    Status
    colourGreen
    titleCOMPLETE

    3.2
    Demonstrate the first working prototype

    Uni Münster server-to-server sharing (i.e. federated cloud sharing) feature has been demonstrated by Holger to Kuba and Peter.

    Image Modified Image Modified

    There was an agreement to

    • plan for a new demonstration between two administrative domains, say SURFnet or CERN and Uni Münster.
    • collect a list of feature requests related to the function (to be discussed with ownCloud)
    • think about trust policies and quality assurances in the context of federated cloud sharing.

    Comments from Kuba:

    • quality of service: in the current owncloud implementation a remote share enters into the discovery process for synchronization - unstable or poorly-performing remote instance may impact users of the local instance
    • authorization: a service manager should be in some control of which cloud instances can issue external shares requests for their users [there is plenty of room for abuse there, in addition this is amplified by the synchronisation of such injected shares on the user’s devices]
    • security: it is not clear how secure the federated sharing mechanism currently is under-the-hood
    • open protocol: is there a bottom-up interest in OCM being embraced by other sync/share software stacks?
    Holger Angenent22 October 201518 November 2015

    Status
    colourGreen
    titleCOMPLETE

    3.3
    Prepare for a demonstration of the federated cloud sharing feature between two administrative domains, say SURFsara and Uni Münster

    We are looking for volunteers with ownCloud server version v.8 or above to test the feature.

    Ron, SURFsara pointed out the the Federated Cloud Sharing feature does not work together with SAML/Shibboleth based authentication. This is a showstopper for a planned SURFdrive - Uni Münster demonstration.

    Ticket has been created: https://github.com/owncloud/core/issues/21227

    ownCloud is working on a quick work-around. FCS just needs a user name.

    Partners to demonstrate Federated Cloud Sharing on 19 January 2016:

    • Uni Münster (local users)

    h_zimm01@uni-muenster.de@uni-muenster.sciebo.de
    a_wilm04@uni-muenster.de@uni-muenster.sciebo.de

    Holger Angenent

    Andreas Wilmer

    Ron Trompert

    David Jericho

    Guido Aben

    Christian Kracher

    Simon Leinen

    18 November 2015

    19 January 2016

    Status
    colourGreen
    titleCOMPLETE

    3.4Initiate discussions about policies, metadata release, directories, legal issues, etc.

    Two main topics have been identified (18 November 2015)

    1. Trust policy: What level of trusted relationship is needed to be established and maintained between two administrative cloud domains in order to share files and folders among their users.
    2. Quality assurance: What service level assurances are needed to be agreed and verified between two server operators in order to maintain the integrity and scalability of shared files and folder inside or outside of the users' file system.

    Mind map (15 January 2016):

    Image Modified

    Kuba at CERN talked about the open issues and Simon at SWITCH talked about the standardization aspects

    (19 January 2016)

    Image Modified Image Modified

    Kuba: http://cs3.ethz.ch/presentations/Interoperability/02Moscicki.pdf

    Simon: http://cs3.ethz.ch/presentations/Interoperability/04Leinen.pdf

    Jakub Moscicki

    Simon Leinen

    All

    18 November 201519 January 2016

    Status
    colourGreen
    titleCOMPLETE

    4. Delivery      
    4.1 Workshop

    Open API v.1.0 documented and released at least in BETA version with the intention to come up v.2.0 vendor agnostic version (IETF WG)

    Cloud Services for Synchronization and Sharing (CS3) Workshop

    ETH Zürich, Switzerland; January 18-19 2016

    http://cs3.ethz.ch/program.html

    Slides and presentation/demo materials are available!

    Image Modified

                  Praying for OCM to work...

    Peter Szegedi

    Christian Schmitz

    Kuba Moscicki

    Simon Leinen

    15 November 201519 January 2016

    Status
    colourGreen
    titleCOMPLETE

    4.2 DeliverableCome up with recommendations for the development of the API towards a new v.1.0 according to the requirements of an open standard.

    Set of recommendations:

    Integration with Macaroons

    Cloud user lookup service

    • Need @ a @ simple @ username @ structure
    • endpoint discovery: DNS

    David:

    • Rather than trying to implement a meta-data or negotiation server, why don't we just use the NAPTR/SRV DNS RR method I proposed late last year? Multiple records, one for each protocol, makes discovery and announcement very easy, and allows the sites to use different endpoints for different protocols as suited.
      Something akin to:

      _standard._opencloudmesh._tcp.cloudstor.aarnet.edu.au. SRV 0 0 5009 ocm.cloudstor.aarnet.edu.au.
      _quic._opencloudmesh._udp.cloudstor.aarnet.edu.au.     SRV 0 0 5009 quic.cloudstor.aarnet.edu.au.
      _ftp._opencloudmesh._tcp.cloudstor.aarnet.edu.au.      SRV 0 0 5011 ocm.cloudstor.aarnet.edu.au.

      ...and so on. Discovery and compatibility is known by simple DNS query, and dynamically configurable using existing standards and tools. Supporting new protocols becomes a server side issue, rather than having to make any additional software aware.

    Protocol negotiation handshake

    Holger:

    • What would you think of a signaling mechanism, comparable to the handshake when handling the method for an encrypted connection? We think of something like: Server A asks server B for a connection and offers WebDAV and CMIS and asks server B which of these it likes. Server B speaks only WebDAV so they agree to use this. Of course it is clear that now it could happen that both servers do not offer a matching method. But the acceptance threshold to participate at OCM could be lowered since only the signaling would be mandatory.

    David:

    • At least if the mandate was WebDAV for the most basic method, that would be the lowest barrier to entry. WebDAV has been around since 1996, and it really doesn't extend existing HTTP that far, there's no real excuse for not implementing it. A technical person is able to talk to a service using telnet/netcat/socat and do sharing functions manually, as well as making it simple to do these things programmatically using curl. Metadata against the query is also trivial, because HTTP/(1.1|2) support headers that can be used to carry all the relevant information.
    • Caching should be a secondary consideration, because it's a non-trivial activity to get right (consider HTTP caching and HTTP 304 return codes). Our primary concern is fast transfer, reliable transfer, and having people join in. In the case of many small files, many alternate protocols allow streaming, batching or the like, akin to the old days of asking a ftp server for a tar of a directory.

    Security and Trust

    • Trusted curcle of servers vs. open internet policy
    • Verification of architectures and deployments up to a community standard. DeiC's Smashbox idea
    • More admin features are in 9.0 also better addressing and web popup. Shibboteh ticket.
    Data path for federated shares
    • pass-through: data is pumped from server B to the client A via server A (as currently available)
    • server-cache: as mentioned by Holger (comes with potential consistency&refresh mechanism issues to be addressed)
    • client redirect: as mentioned by Paul Millar (would affect the current usability model of ownCloud)

    Pick a preferred home for standardization

    • IETF??? - Simon to give a thought

     

    All the recommendations have been fed into the Phase II initiation phase.

    All19 January 201605 February 2016

    Status
    colourGreen
    titleCOMPLETE

    5. Closing Summary

    The OCM team agreed to continue and split into two working groups (WGs):

    1. Strategic policy and standardization oversee WG
    2. Technical protocol definition and implementation WG

    The key objective of the Strategic WG will be to reach the ultimate "Open Cloud Mesh standard" and to oversee (not overdrive) the Technical WG making sure that the open API design principles are properly addressed. The key objective of the Technical WG will be to deliver one or more working prototypes of OCM and to provide (not force) input to the Strategic WG making sure that their assumptions are realistic.

    The detailed objectives and key results of the Phase II will be determined in the initiation phase.

    Phase I is considered to be CLOSED.

    News item: http://www.geant.org/News_and_Events/Pages/OpenCloudMesh.aspx

    Peter Szegedi19 January 201605 February 2016

    Status
    colourGreen
    titleCOMPLETE



    History

    • Around early 2012, TF-Storage participants started to actively look into data storage software platforms in order to provide on-premise file-based sync&share (aka. Dropbox-like) services to their constituencies.
      • Some NRENs even ventured into the development of a proof-of-concept tool called the Trusted Cloud Drive (TCD) under TERENA
    • By mid 2013, ownCloud appeared to be the most promising one with a growing open-source development community behind.
    • In December 2013, the GÉANT Association (formerly known as TERENA) and ownCloud Inc. made an agreement that serves to facilitate the desire of various National Research and Education Networking organisations (NRENs) to introduce services based on ownCloud technology and/or to offer the technology to their constituencies.
    • As part of this collaboration effort, in January 2015, Christian Schmitz from ownCloud initiated an idea (aka. Open Cloud Mesh) to interconnect the individual on-premise private cloud domains at the server side in order to provide federated sharing and syncing functionality between the different administrative domains.