Weblog

Package Object Types

Created by Dave Bauer, last modified by Gustaf Neumann 02 Jun 2017, at 02:29 PM

This describes the discussion for TIP#120 https://openacs.org/forums/message-view?message_id=1482643

APM Packages are all of the same acs_object_type of 'apm_package'.

I propose creating an acs_object_type for each package type as a subtype of apm_package.

This would allow for a hierarchy of package types. The most common example we might think of is dotlrn. The dotlrn package could be a subtype of the subsite package. This would allow an easier way to determine if a package takes on the role of a subsite type package and it could greatly simplify all the code that has to be special-cased to determine if dotlrn is installed.

This would also allow simplification of the case when a new package is created to customize the user interface of another package. For example, many people would like to customize the forums package, to change the style of user interface. If we had a custom-forums package type that was sub-typed from forums, we could have a simple and consistent mechanism to defer pages to the parent package type for handling. This would allow much easier reuse of pages and would make it very simple for local customizations to be handled safely without worrying about accidentally adding custom code back into OpenACS.

 The immediate goal of this work is to make the dotlrn package a subtype of the acs-subsite package. I have done preliminary work to turn dotlrn communities (groups) into application groups of the dotlrn package also. This should make it easier to do an upgrade to an existing dotlrn install that will support subsite based dotlrn.

Previous work includes the https://openacs.org/api-doc/proc-view?proc=subsite%3a%3apackage%5fkeys subsite::package_keys procedure. This basically is a hard coded list of packages that fulfill the subsite role. Defining children of a subsite object_type would allow greatly flexibility and use the existing features of the toolkit.

 Steps to implement

1) Make a new object_type for existing package_types when a new package type is created. Default is subtype of apm_package.

2) Add parameter to package create code to specify package super-type

3) Add tag to info file super-type which is optional and will default to apm_package. This tag will have the same behavior as requires element so Tcl libraries are loaded of the super-type.

4) Add attribute to apm_package_types for inherit_ui_p to specify if URL resolution will go up the package type hierarchy.

5) Update apm_paramaters to support setting parameters for the super-type package parameters up the package type hierarchy. Allow sub-typed packages to override the default value of an inherited parameter.

6) Fix package loading to load Tcl libraries in package dependency order.

Comments by Don Baccus:

Let's think about UI issues for a moment ... I have an idea to toss out

Currently, abstract URL resolution works like this:

 1. If a file or template at [acs_root_dir]/www/${url_path} exists, return it

2. If a file or template at [acs_root_dir]/packages/[ad_conn package_key]/${url_path} exists, return it

3. 404

 We can extend this if we allow package sub-typing ...

replace 2 with ...

2. set package_key [ad_conn package_key]

2a. while no file exists at [acs_root_dir]/packages/$package_key/$url_path do

2b.   if parent type is apm_package, break, else set package key to parent type's package key

2c. od

2d. if exists - return file or template

In other words, allow for the inheritance of parent package UI.

 Often, of course, this isn't always desirable, so an apm package type would want to have a "don't inherit UI" attribute.  You'd do this, for instance, if you were going to brand new UI for an existing package.  Say ... a portlet.

Inheriting the UI though allows for some things.  For instance, dotlrn could be mounted at /, no need to mount an acs-subsite instance at all as is done now because its UI would already be available (assuming dotlrn was derived from acs-subsite).

How would this impact performance?  Not at all, if the request processor's performance mode is enabled, and when it's not, who cares?

Comments by Gustaf Neumann:

Defining package types as acs-object-types and allowing sub-typing/sub-classing for packages using these is a good idea. As presented at the OpenACS conference in Guatemala, xotcl-core and xowiki support this already since about a year. So is it possible to define the s5-package by reusing xowiki in a few lines of code. Other examples were presented by Victor Barrios (mashup) or Stefan.

Subtyping acs-object-types is just the first step, the (desired) implication are something which might need more discussions. Some changes are desirable for the package manager or the request processor.

Note, that in the xo* packages, it is not only the case that packages are object-types, but that package instances are objects (actually xotcl objects), which provide the package instance specific context for a request. Package instance objects are initialized at the begin and deleted at the end of a request.

Below i will use the term "reused package" for the superclass (the more general package) and the term "specialized package" for the subclass (the more specific package, inheriting from the superclass package).

Here are a few items from my experience:

1. package loading: if a package reuses an other package via sub-classing, it should load the library files (.../tcl/*tcl) of the sub-classed package as well and initialize it. Daveb, why not use requires tag in the info file? If there is more than one package reusing the same package the library is loaded more than once.
2. package parameters: a specialized package should inherit the package parameter definitions of the reused package(s)
3. per-request files (../www/*) and context management: it is desired/necessary to share some of the per-request files and to allow a per-package-instance initialization (constructor)

For the three issues, the xo* packages use the following strategy:

• package loading:  
  Since the loading order of apm-packages at boot-time is mostly alphabetical, there must be a way to say to load required packages. In particular, the specialized package should load the library files of the reused package (e.g. s5 should load the library files of xowiki) Daveb Would it make more sense to just fix packages to load in required order?
   
  • Working solution in xo*:
    Add a line like the following to e.g. the first loaded file of the s5 package (the specialized package) to indicate the dependency of xowiki (the reused package)
    
    ::xo::db::require package xowiki
    
    This call checks, if the library files of the specified package are already loaded and load it via apm_source, if not (see as well http://alice.wu-wien.ac.at:8000/s5-xowiki-talk-guatemala/presentation?slideshow=1&pagenr=5)
     
  • Desired solution:
    While the working approach works fine with moderate effort, it can be improved via the apm-package manager. It should be possible to specify in the package manager "this package is a subclass of some other package" an add the specialized package automatically to the dependency list. The apm* procs should sort the dependencies in a topological order and load the packages this way. There would be no need to figure out what the first loaded file is, since the order is on the package level. Daveb yes APM needs to have support to specify when you create a package. Roc The topological load order is desirable, but at this moment standard OpenACS mostly just loads the libraries that's why we don't see problems often, while in xo* packages you actually use it for construction, right? (that was the problem I detected). Will be good to use topological for subtypes and package dependencies.
     
• package parameters:
  Without any support, it would be required to replicate the definition of the package parameter of the reused packaged in the specialized package. Extending/altering the package definitions in reused package would have to be repeated as well in all specialized packages (maybe over some specialization levels)
   
  • Working solution in xo*:
    xotcl-core has an object oriented package parameter code that searches parameters on the superclass hierarchy of the specialized package. This leads to several space and speed improvements (see e.g. http://alice.wu-wien.ac.at:8000/s5-xowiki-talk-guatemala/presentation?slideshow=1&pagenr=6 and following pages). The interface of the oo-code mirrors the interface of the classical interface (e.g. parameter get). The implementation does actually more than just searching along a path, it avoids redundancy as far as possible)
     
  • Desired solution:
    The current implementation allows just to share package parameters, if the specialized package actually wants to overwrite some parameters, it cannot do it, since it uses the old shared/parameters?package... scripts, that are not OO aware. So, currently, to alter parameters they have to be copied or the shared/parameter code has to be adapted.  It should be possible alter the default value of a parameter in a specialized package for further inheritance or other instances of a reused package. It is as well desired to add permissions, maybe, not every package admin should be allowed to alter all package parameters (e.g. changing policies in xowiki in a dotlrn-like installation, where package-admins are teachers/lecturers/...) Daveb I think there is a good proposal around to make this work by scoping parameters at the package level or package instance level that would help here. Permissions is an interesting idea but probably more complex than we need to get started. Roc Will be good implement parameter sharing / reuse (and less redundancy) for non-xo* apps.
     
• per-request files and per-instance context management:
  The specialized package will share some files from the reused package (e.g. www/admin), but it also wants to perform some package specific initialization
  • Working solution in xo*:
    the specialized packages use index.vuh files for www and www/admin, all per-request files use ::pkgkey::Package initialize to resolve the context of a package instance and to call the package initialization (see e.g. http://alice.wu-wien.ac.at:8000/s5-xotcl-core-tutorial/presentation?slideshow=1&pagenr=66 and http://alice.wu-wien.ac.at:8000/s5-xotcl-core-tutorial/presentation?slideshow=1&pagenr=79 from the xotcl-core tutorial).

    An example for the redirector is https://raw.githubusercontent.com/openacs/s5/master/www/admin/index.vuh

    The approach with index.vuh works quite well, since physical existing files override the virtual (index.vuh) redirect. So, all per-request files in www/* in the specialized package, which should be different from the reused package, can be put to the www/* directory of the specialized package (e.g. s5). Not sure, how this approach works over multiple specialization levels.
     

  • Desired solution:
    As Don points out, the search for files could be done automatically by the request processor in case the package is a specialization of another package. This approach has no problems with multiple specialization levels. One has to sort out as well the interactions of .vuh files and package hierarchy search. Roc The request-processor solution is the best and cleaner.
    
    It is as well desirable to call from the request-processor the package specific initialization DAVEB. What is a package specific initialization? This seems to refer to xowiki based packages that have initialization procedures that setup the context for the request. I believe this is not a core feature yet.

Migration from CVS to GIT

Created by Victor Guerra, last modified by Gustaf Neumann 02 Jun 2017, at 01:17 PM

IMPORTANT!!! by no means the statements written in this page are official. At the moment one should consider this information just and only for experimental purposes. There are no official decisions made by the OCT so far regarding migration to GIT or any other source control tool.  At the moment CVS is the official tool used and you can navigate the project history and code at our official code and history inspector Fisheye OpenACS

Tools to consider for the migration:

• GIT includes a tool for the migration of CVS repositories called git-cvsimport.
• The cvs2svn project provides support for converting a CVS repository into GIT. The tool is called cvs2git

Structure of the GIT Repository:

• One git repository hosting the entire code base of OpenACS (including contrib and all packages) vs. multiple repositories (provide separate git repositories managed via modules/subdirs/...)
• A single large repository seems not realistic, since many sites have site-speciifc modules, since there projects like ProjectOpen and others having developed a long range of maintained packages. But are 300+ separate repositories manageable with reasonable effort (maybe mr is currently the best option, we have good experiences with this). However, this decision has large impact on tagging and release management (see below)

Where could the repositories live (?):

• Official repositories on openacs.org machine? Having remote-mirrored repositories on github so developers can clone and share patches using github. -- jiml: many tools which could get similar function to github (forking, pull requests),,,  can we put up a gitorious on a test site with maybe 50gb of repo space so all the committers can have their own?
• Using github.com as our official source code site? or Bitbucket?

Points to consider before | when migrating to GIT: 

• Release-management: Installing-software-from-repository code is based on release "channels" and cvs-tags in order to define which channels are available to users ( https://openacs.org/repository/) and to provide the recent packages for that channel.  In principal, similar tags could be used as well in git, but there are several differences. CVS tags are per file, git tags are per repository (if the git repository contains e.g. all packages, there is no way to tag a single package as e.g. OpenACS 5.9 compatible with e.g. tag "openacs-5-9-compat"). It is possible to work with separate git repositories and work with git modules and/or subtrees, but this means that we have 300 repos to manage (all could have separate branches and tags), and single commits covering multiple repos are not supported). This means, that one needs higher level tools and recommended workflows to manage developments on the core source repositories (branches, tags) and site-specific packages and local modification as well as site-specific release and feature branches
• Depending on the decision of the structure it is necessary to provide git-guidelines similar to cvs-guidelines and the code retrieving and generating .apm packages has to be rewritten.
• Application for voting on oct-elections depends on commits history ( most likely this will transparent since we use fisheye.openacs.org  for fetching information about commit history, and fisheye provides one and one only interface for querying data, so the fact that the code is being controlled by cvs or git is irrelevant ... but since fisheye is configured for CVS, it has to be newly built for the resulting infrastructure, or it has to be dropped/replaced by github/bitbucket or others.

Current state:

• Since 2014 Victor Guerra (Vienna University of Economics) has created an unofficial mirror on github https://github.com/openacs which contains a daily mirror of the CVS repositories at openacs.org. It consists of an openacs-core modules plus a separate repository for every non-core package.

WCAG 1.0 Checkpoints

Created by Emmanuelle Raffenne, last modified by Gustaf Neumann 02 Jun 2017, at 09:06 AM

Status at 21 october 2008 (OpenACS 5.4.3 and .LRN 2.4.1)

A summary for US Section 508 is available at LINK_TO_WIKIPAGE 

See Checklist of Checkpoints for Web Content Accessibility Guidelines 1.0 for details

How to read this document

Accessibility Compliance Level

• Level A means that all the priority 1 checkpoints are in state "Y" or "n/a".
• Level AA means that level A has been reached + all the priority 2 checkpoints are in state "Y" or "n/a"
• Level AAA means that level AA has been reached + all the priority 3 checkpoints are in state "Y" or "n/a"

Checkpoint Status

• Y: 100% of the pages comply with the checkpoint or the exceptions are clearly identified and listed in the accessibility statement.
• N: NOT 100% of the pages comply with the checkpoint, exceptions are NOT identified.
• n/a: the situation described in the checkpoint is not applicable in ANY of the pages

OpenACS-specific Techniques for Checkpoints

Each checkpoint is identified with a number which links to the OpenACS-specific techniques to apply it.

Note: the  links are dead right now, but will point to OpenACS-specific techniques to implement them.

Priority 1 checkpoints

Priority 2 checkpoints

Priority 3 checkpoints