Denton log: 1996

1Q96

plan:

distributed debugging; algebra applications; statistical computing; virtual shared objects; programming environment; performance analysis and tuning;

actual:

Meeting in Southampton on 960124. Present: Dave, Luc, Russell, Julian, Rob. Joined by Richard ?, Danius and Johnathan (Dale).

MBONE:

Both sites have the facility for MBONE up to the desktop and we will shortly experiment with voice and shared whiteboard. Richard demonstrated MBONE delivering video around the lab in Southampton. MBONE broadcasts over UDP and as such is unreliable (ie. packets may not get received, but this does not matter crucially in video applications). Dave was keen on identifying ways in which MBONE could be used for inter-site communication by distributed programs (as an alternative to http tunnelling), but the conclusion was that while it could just have a role in distributing work in a processor farm, the complexities of dealing with non-guaranteed delivery in the application far out-weighed any advantages.

Outputs:

Luc's paper for ICFP accepted. Dave + Christians' paper for ICFP accepted. Julian's paper rejected. Luc, Dave and Julian have revised their respective papers for PSLS (Beaune). Dave and Johnathan are writing for PAAM and (two) for SDNE. Julian also considering writing for SDNE (deadline Jan 29). Luc also hoping to write about CEKDS for Europar '96, deadline Feb 18. Dave and Danius have written a tecnical report about the ALCD work and it will subsequently be submitted to a journal. Luc has written a memo for the project about his work (he also presented this at the meeting).

Technical progress:

• Andreas has released version 1 of youtoo, and there is also an alpha release with an interactive top loop. He has also built a Sun4/Solaris version of youtoo release 1 (primarily to support the final year projects taking place in Bath). This had the beneficial effect of revealing some bugs.
• Dave has started work on the sockets library and predicts beta release in early February.
• A linkbase for the EuLisp definition will shortly go on general access (demo service). Arising from this, the principal project members present agreed to help Julian in producing a complete HTML version of the definition. Luc's interface from emacs to the distributed link service has stopped working with the move to 19.30.
• The matter of compiling youtoo for Windows was discussed: it is believed that Boehm-Weiser does not work under 3.1 or 3.11 and the suggestion was to go for NT, which should also be valid for '95. However, it may also be worth substituting a different GC (Bartlett's?) or leaving it out altogether.
• The loan of an ether switch in Bath has been delayed by a shortage of long UTP cables for connecting the Indys.
• Julian has built the MPI library for the Indys. Youtoo + MPI may be a suitable environment for Danius to produce the statistics demonstrator on the SP2. Julian has also ported Christian's pattern matching compiler to EuLisp to support the work on compiler and executor generation.
• Final year project students (Bath) are looking at adapting existing C header file processors to generate EuLisp modules with extern declarations.

Actions:

Julian:

pre-process remaining sections of EuLisp definition and distribute to Dave, Luc, Russell for completion.

Rob + Dave:

liaison over tunnelling through http via cornucopia for inter-site communications.

Dave:

reimbursement of Julian for train tickets from Beaune meeting.

Julian:

ask Jill to send Southampton an invoice for kronos.

Julian + Dave:

facilitate MBONE interconnection.

Next meeting:

will take place in Bath on February 15, 1996. Items to carry over: discussion of the relationship between Java and Denton and a report from Dave on his meeting at MIT with Stephen Adams.

No record for meeting of 960215

No record for meeting of 960228

2Q96

plan:

distributed debugging; algebra applications; statistical computing; AI applications; distribution;

actual:

Week long meeting in Philadelphia 960520-26. Present: Russell, Rob, Luc, Julian. General discussions about future directions and applicability of research presented at the meetings attended (PADS, PLDI, ICFP).

Meeting in Southampton on 960619. Present: Dave, Luc, Russell, Julian, Juan. Joined by Danius, Benoit Lanaspre, and Tim Chown.

Outputs:

Dave's papers to SDNE and PAAM and Luc's paper for EUROPAR'96 were accepted.

Technical progress:

• Juan described the application he is building with youtoo. His program models a Spanish fish market (with a descending auction); it is a multi-agent system using MPI for communications between agents. MPI is interesting for the point-to-point communications, collective operations and process groups. Disadvantages are that, process management in MPI is static (no primitive for creating processes at run-time), changing communicators configurations is a major operation requiring barrier synchronisation and debugging is difficult. The second version of MPI should solve some of these problems, and XMPI - X graphical interface for MPI - should facilitate debugging.
• Andreas is working of the next version of youtoo, soon to be released. It includes threads, a number of bug fixes, and is faster (startup, generic dispatch, ...)
• Dave first reported on his trips to the SDNE workshop and the WWW conference. It appears that the WWW is more and more considered as a transport layer for distribution, not only for documents, but also for objects, programs, tasks, ... Then he described the persistent object store that he started to design based on the following requirements: 1) need of communications between Bath and Southampton 2) WWW is the transport layer 3) provide a persistent store.
• The matter of interaction of threads and garbarge collector was discussed. Luc is going to inquire about a port of Boehm's gc to Posix threads and NT.
• Luc presented his work on dynamic binding.
• Danius reported on the port of the GNU bignum package too youtoo.
• We concluded the meeting by a discussion on the strategy for the rest of the project.

Actions:

Luc

inquire about portability of Boehm's gc on other platforms

Dave

persistent object store

Next meeting: In Bath, probably either July 11 or 12.

Meeting in Bath on 960712. Present: Julian, David, Ludo, Andreas, Julio, Juan, Enriche, Rob, John, Russell.

Technical reports

• Andreas out-lined his interface to MPI, which works via the <stream> class, thus a process reads and writes data to receive it from or send it to the network.
• Juan gave an update on the development of the Fish market simulation.
• Julio described the Tcl/Tk object hierarchy and a particular problem in passing arguments to callbacks. Discussion identified a solution based on scanning strings at run-time, but subsequently he found a neater solution.
• Enriche reported on the FFI generator, which he has working for basic types, but as yet, he has no general solution for pointer types.
• Ludo sketched his work on the static and dynamic analysis of instructions sequences in Youtoo. Andreas will be using this to improve the peephole optimiser phase of the compiler. In the meantime Ludo will begin on translating the bytecodes to threaded code using gcc's labels.
• Julian described the agent architecture he built for the fourth of the fish market simulations, reported progress on the specification of garbage collection (joint work with Luc) and on his visit to Kiel to discuss compiler correctness.
• Dave addressed three items: (i) Nexus, investigations are continuing with the aim of using it as a compiler target language (ii) the PSTPA meeting (boring, but worth it to hear EPSRC news early) at which the new themes of emergent computing, distributed information management, distributed architectures, multimedia process architectures (iii) the April language developed at IC by Keith Clarke and Frank McCabe.

Actions:

None.

3Q96

plan:

distributed debugging; algebra applications; statistical computing; AI applications; distribution;

actual:

Meeting in Southampton on 960906. Present: Andreas, Dave, Julian, Luc, joined by Danius, Antonina and Walter

Technical progress:

• Andreas and Julian described progress about the implementation of youtoo. First, FFIGen was ported to youtoo by Enric Hernandez: FFIGen parses C syntax using the front end of lcc, then generates a module definition for the youtoo module language. Two issues remain to be addressed: FFIGen is implemented in EuScheme and int* references are not handled properly. Second, Julio Garcia defined a tcl/tk binding for youtoo. It is interesting to observe that threads were used to handle several inputs coming from the read-eval-print loop or tcl/tk buttons. A long test suite ran successfully. Third, Andreas defined a MPI binding for youtoo. Migration of functions (and code) is going to be supported soon: Andreas explained the technique he intends to use for the implementation. Fourth, libraries are now shared in youtoo. As a consequence, a youtoo image is now only 20Kb, while it used to be 2.5Mb (this includes the gc, level1, virtual machine, compiler, dynamic linker).
• Four different versions of the fishmarket problem have been implemented by Juan. 1) C+PVM 2) C+MPI 3) Java 4) youtoo+MPI+TCL/Tk. The latest version assumes a fixed number of buyers, sellers, auctioners, ... A fifth version is expected to be implemented in the near future: it will support a variable number of "agents" (decided at runtime), using migrable youtoo threads.
• Luc reported on the port of Nexus to PPCR. This was followed by a discussion about the different semantics of threads as well as their interaction with exceptions and continuations. According to PPCR, sequential programs can be recompiled and can run safely in a multi-threaded environment: this implies that file descriptors and signals are per thread and not per process. On the contrary in Nexus (or Posix threads), file descriptors can be shared between threads because defined at the process level. As far as control in a threaded environment is concerned, Eulisp has opted for a minimalist philosophy motivated by portability and efficiency considerations. Consequently, other behaviour for control structures should be defined in higher-level layers, e.g. like the semantics of future.
• Dave reported about his work with Stuart Maclean about the conversion of Prograph internal format to an S-expression. In the future, an evaluator is going to be written to execute Prograph programs in parallel.
• Dave reported about the sound farm on the sp2 using the Harvest Cache. The report was followed by a discussion about building a system using URL as global pointers. In particular, we debated over the best technique to create URL global pointers in youtoo. The solution consisting in intercepting every call to gc_malloc was rejected because too low level. We thought it was better to use the object system. A first solution is to define a new make method, but the most elegant solution is to define a new meta-class. An issue that was also slightly addressed is the time at which the global pointer should be created. The "eager" solution creates global pointers at allocation time, while the "lazy" solution creates global pointers only when the object is migrated. At the moment, it seems difficult to decide which one is best, because we have not yet defined the requirements for such a system.
• We concluded the meeting by a discussion about Denton III, the followup to this project. We do not want to reject a proposal which is "language-specific" only, but we discussed the possibility of an "application" project that would use the tools developped in Denton II. Several areas were suggested like business process control (Peter Henderson coordinating this theme), artificial life, computational biology (someone working in this area in Bath).

Actions:

Danius

send the bignum package.

Andreas and Dave

exchange their socket libraries.

Julian

find the integer factorisation as a test suite for the bignum package.

Julian or Andreas

send the scheme-compatible module to Luc

Luc

study the requirements for a WWW-nexus like system.

Next meeting:

not decided, we are waiting for Julian and Dave's timetables.

4Q96

plan:

distributed debugging; algebra applications; statistical computing; AI applications; distribution;

Actual:

Meeting in Bath on 961016. Present: Julian, Russel, Andreas, Rob, Dave (Power), Francisco, Dave (DeRoure), Luc.

New Members:

David Power is a new postgraduate student supervised by Russel. He is working on parallel computing and security. He might also be involved in some computer algebra.

Francisco is visiting Bath for a period of 3 months. During his stay, he is going to work on two different tasks. He is going to implement the Fish Market (Version 5) where places (in a fixed number) are represented by processes, and agents, represented as threads, will migrate between places. He is also going to port NOOS to youtoo; NOOS is a knowledge system written in Common Lisp, and could be the AI demonstrator. Foreseen problems concern the use of set on dynamic variables.

Reports:

Andreas reported on the OOPSLA conference. He attended two tutorials: developping GUI with Java AWT (Abstract Window Toolkit) and OLE, CORBA, Infrastructure for distributed computing. The latter tutorial seems to be particularly relevant to our project. Besides numerous acronyms to remember (DCE, OLE, RPC, SOM, IBM, NEO, SUN, COM, IDL, JOE), it would be interesting to understand our position with respect to the CORBA approach.

Julian reported on the VIM meeting in Barcelona and on the discussions he had with the Commission in Brussels for an 9 months extension. Please, refer to the VIM minutes for more details. A workshop will be organised in May 97. There is a call for volunteers to organise the meeting, create a program committee, and find a publisher.

Julian described the ATM configuration in Bath, while Dave described the new configuration after the move, in Southampton.

Technical progress:

In Southampton, an effort is made to design an agent architecture that could be used by several projects involving agents. In order to evaluate the genericity of the architecture, we had a discussion on the primitives used to control agents in the different implementations of the Fish Market. There are primitives to communicate by sending and receiving messages, to dynamically create agents, to migrate (the behaviour of) agents. (Not all implementations support these primitives, especially the MPI implementation does not allow us to create processes dynamically).

Luc briefly described the primitives he designed with Christian Queinnec to control computations in a distributed environment.

Luc described the status of Nexus-Scheme. The Nexus library was ported to the PPCR thread package, and Nexus primitives were made accessible in Bigloo. A document distributed during the meeting describes the API available. On top of those primitives, farm processing and task stealing facilities are also offered. Luc is going to fix the remaining problems (memory leak and thread-safe continuations) before spending a few days in Bath early December to port the code to youtoo.

The current API of Nexus Scheme was also proposed as an API to program over the WWW (and http protocol), with global pointers being represented as URL's. Related to this task, Julian has proposed a 3rd year project, to implement a binding of youtoo with the Harvest Cache. It is not clear whether the Harvest Cache will offer the appropriate communication model for the Fish Market application. Indeed, typical communications between agents are point to point, and in such circumstances, the cache, which acts as a passive persistent store, becomes useless.

Actions:

Dave

prepare a telnet tunnel Demo. The idea is to have two processes (one sitting in Bath, the other in Southampton) talking together through the telnet port.

Programming over the WWW

we delay the design of the Harvest-Cache-based memory model and programming model over the WWW, until next year, when Dave is on sabbatical.

Next Meeting:

Pencilled dates are Fridays November 15th and 22nd. Julian and Luc to tell their availabilities. Most probably, meeting will be held in Southampton so that Dave does not have to cancel lectures.

Meeting in Bath on 961016. Present: Julian, Russel, Andreas, Rob, Dave (Power), Darren (Ward), Peter (Bradshaw), Dave (DeRoure), Luc.

New Members:

Darren (Ward) is a new postgraduate student supervised by Julian. He is working on virtual robots.

Peter (Bradshaw) is an undergrad student supervised by Julian. He is working on MUDs (Multi User Dungeons)

Reports:

Luc reported on the status of the port of Nexus to Youtoo. A few problems remain to be fixed in youtoo in order to be able to execute some demo code.

Luc and Dave reported on their experience using Youtoo and Eulisp. Details appear in the next Section.

Dave described the SUD (Single-User Dunjeon) that he has implemented for his Lisp course, and explained how it could be extended to a MUD.

Experience with Eulisp and Youtoo:

In Eulisp, evaluation order is not specifed at the toplevel of a module, but the Youtoo interpreter specifies the evaluation order: for instance, in the interpreter, it is not valid to evaluate (lambda () a) before the variable a is defined.

It seems possible to redefine special forms in Eulisp, but it is not the case in Youtoo. Indeed, Youtoo has an "expansion order", successively dealing with special forms, macros, and then lexical variables. With a such a program, (defmacro if (p t . e) `(if ,p ,t ,(if (null e) #f (car e)))) we should be able to define a new if macro that refers to the special form if. Scheme defines a notion of scope for macros with let-syntax and letrec-syntax.

Eulisp module system is very convenient to specify the modules that should be used at macro expansion-time (as opposed to Bigloo for instance, which does not provide any import statement for macros).

On the other hand, a macro cannot be used in the same file as its definition. Luc consider this problem to be inconvenient, while other members find that it is useful to distinguish compile-time and run-time code. This approach has a serious inconvenient when writing macros that generate macros, because it requires to introduce a file (i.e. a module) per level of expansion. This observation lead us to think that it is not possible to compile Christian Queinnec's macro tower which requires an unbounded number of expansion levels.

Though "stylistic" reasons seem to forbid the use of a macro in the module of its definition, the absence of an eval function make it impossible to implement it in practice. A solution would be to write a full interpreter for Eulisp, but there is a risk of inconsistency between the interpreter and the compiler. The other solution is to define eval as the execution of compiled code. Andreas mentioned that such a facility would complicate the compiler which would have to perform the same pass several times. Luc thought that this was "just" a matter of writting a fully reentrant compiler; this might not be efficient but it should work.

Youtoo acts as a compiler and as a make. Dave and Luc would like to be able to turn off this make option, because it does not allow parallel compilation (as the same file could be recompiled twice in parallel).

In addition, there might be more dependencies than the ones detected by youtoo. For instance, Luc wrote a include macro, which was inlining the code contained in a file at macro expansion time.

Modules are not allowed to be recursive. This is a serious constraint which seems to have been encountered by Dave, Julian, Rob and Luc. This problem can be avoided by moving some variables in a parent module, or by including several files into a single module. This leads to a non-natural program organisation or prevents separate compilation.

Julian said that Eulisp definition concerning modules contained statements that were fully understood, but could be extended.

Luc mentioned a feature of the Bigloo module system which was convenient: Bigloo distinguishes importing a module (which states where free variable can be found) from using a module (which initialises a module). Initialisation of a module is done only once, but imports can be multiple and recusive.

Actions:

Andreas

to fix Youtoo problems to be able to compile the Nexus interface to EuLisp

Luc

to test the new version of Youtoo before the official release.

Next Meeting:

Pencilled week is January 20th to 24th. If Luc goes to the JFLA, he will be back on the 22nd only. Andreas will be in Pisa.