The focus of this meeting was to think about the future of EPICS. As I recall there were about 6 to 8 people who attended the meeting. I also recall that each person offered at least one vision and normally more than one.
At the end of the meeting I remember thinking:
How can we possibly mesh all these visions?
In December 2005 APS asked for people to volunteer to leave ANL. The offer included up to six months severance pay.
I had been thinking about my vision for the future of EPICS ever since the July meeting. I also thought that, if I stayed at APS, I would not have time to implement my vision. I took the offer and left ANL and started work on the vision.
When I left APS, I thought it would be about three years before I had users.
Obviously I made a really bad estimate!!
A brief statement of the vision was to implement an IOC similar to a V3 IOC but with support for fully structured data. This included the components:
The initial implementation language was chosen to be Java. The reasons were:
pvData defines both introspection and data interfaces for the following types:
This is network support for transporting pvData.
Just like EPICS channel access, pvAccess provides network communication between a client and a server. The difference is that pvAccess uses pvData as the data definition.
pvAccess implements the following interfaces:
A pvIOC is a memory resident database of PVRecords. Each PVRecord has a name and a top level PVStructure. A record is "smart" because it has code associated with it.
pvIOC implements what today is called pvDatabase. In addition it had other, currently abandoned, components that are briefly described at the end of this talk.
When I left ANL (Feb 1, 2006) , we (my wife Irma and I) moved to Osseo Michigan. I made one of the rooms in our house into an office.
The rest of the history through 2009 will be described via a brief description of what was reported at various EPICS Meetings.
By this meeting a lot more was implemented. It had support for pvDatabase and a channelProvider for pvDatabase.
At this meeting Matthias Clausen (DESY) and Bob Dalesio offered to provide financial support for javaIOC development.
For the last half of 2007 through the first part of 2015, I was paid to work on the javaIOC and then on EPICS V4.
Matthias provided initial support (about 1 Man Year). Bob, via DOE SBIR grants and BNL, provided the remaining support.
A major missing component for the javaIOC was network support for pvAccess. During the EPICS Meeting Matthias, Bob, Matej Sekoranja, and I had a private meeting. During this meeting we discussed making Matej responsible for the network support for pvAccess.
Matthias had Matej and me come to DESY for a couple of days. During the day Matej and I met with Mathais and other people from DESY. After the daily meetings, Matej and I took a long walk and then had dinner together. We got to know and respect each other. Matej and I became close collaborators. From then until about the end of 2010 Matej and I were the main developers for pvData, pvAccess, and pvDatabase.
The support for Matej, who works for cosyLab, was initially from Matthais and Bob. In recent years he has also received support from DLS, PSI, SNS, and ESS.
By this meeting even more of what was descibed in the April 2007 meeting was working. This included the first version of network support for pvAccess.
Going back to the July 2005 meeting, remember that many visions were presented, not just mine.
Amoung these was Bob Dalesio's vision. Bob has presented his vision at several EPICS meetings so let me just say that an important part of his vision is Middle Layer Services between clients and data. An important data source is IOC records.
In order to provide really efficent Services it is desirable to be able to run them as part of the IOC that has the records. This means that pvData, pvAccess, and pvDatabase must be implemented in C++ as well as Java.
When Bob said that we must provide a C++ implementation my thoughts were:
Deciding to provide a C++ implementation required some pvData extensions.
Scalar types for unsigned integer types of length 8, 16, 32, and 64 bits were added.
Adding these two new types arose from work at Diamond on areaDetetor. The issue was to provide a pvData implementation of the NDArray used by areaDetector. Several approaches were tried but all were ugly. Union and unionArray provided a nice solution. The NTNDArray normative type (implemented by David Hickin) is the pvData implementation of NDArray.
Whenever I made the statement that an arbitrary stucture of data of the basic primative types could be created via pvData, Bob would say:
We MUST somehow limit the types in order to implement services.
The solution is normative types. A normative type is a structure with a well defined set of fields.
At the beginning of the C++ development C++11 was not yet a standard and I was not familar with Boost C++. It was obvious that some sort of reference counting memory management was needed. My first effort was to implememt it myself. This led to several months of wasted effort.
Then Michael Davidsaver convinced Matej and me to use shared_ptr from Boost. Michael also gave us lots of help with implementation for both pvData and pvAccess. Michael also implemented the code to enforce "Copy On Write" for arrays in pvData.
In addition to Matej and me, the following people have managed and/or developed code that is in EPICS V4 release 4.6:
David Hickin, Michael Davidsaver, Ralph Lange, Andrew Johnson, Greg White, Sinisa Veseli, Guobao Shen, Timo Korhonen, Kay Kasemir, Steve Hartman.
Only minor changes since 2010
Two major enhancments since 2010 are:
pvDatabase implements database of "smart" records via a process method attached to each record. In addition it implements a channel provider for accessing the records.
Since pvDatabase is now implemented in C++ as well as Java, a pvDatabase can be part of the same process as a V3 database.
A lot of effort since 2010 has involved interoperability between V4 and V3.
Channel provider for connecting to V3 records.
Python support for pvData and pvAccess.
A synchronous interface to the client side of pvAccess.
A set of examples for using pvData, pvAccess, pvDatabase, and pvaClient.
Analogous to the "EPICS Application Developer's Guide".
Each field of a record instance can optionally have code attached to the instance.
Code similar to the V3 calc record.
An expression variable can be a Java primitive or String.
Several operators are supported.
Database and pvAccess links to other records is supported.
XML can be used to create PVRecord instances.
Support is provided to attach code to a record.
Similar to scan threads in V3 IOC.
This is a Java implementation of asynDriver.
Support was implemented for the vxi11 protocol (successor to GPIB protocol). This means that a large set of network based hardware devices are supported.
Support was implemented for IP itself and for serveral IP modules.
Support was developed for interfacing to BACNET devices.
This is a "probe" like tool for pvAccess/pvData.