HPCCSystems Internship

In third week, we achieved following goals : Basic Structure for HTML Documentation. XML Doc was read to get appropriate information for HTML We use Jinja2 library in Python to generate HTML doc from XML. Use Bootstrap and jquery for CSS and JS. The documentation in HTML was divided into 3 regions : IMPORTS sections : here we link to imports in ecl file. Definition Tree : In this section, we simply show the hierarchical structure of ecl code in form of an nested collapsible list. Descriptions : here we show each attribute with their documentation. Definition Tree was linked with description and vice-versa

In second week, we achieved following functionality : Basic structure of ecldoc utility was generated in python. Python utility uses a configuration file as input that specifies : Input root of documentation - where we start processing the files Glob patterns wrt to input root for files to be processed. Output root - where output documentation will be stored. format of documentation required. Glob pattern were parsed using glob utility in python 3. LXML lib was used for processing XML in python 3. XML documentation was generated for each files matched by glob pattern. To gen XML documentation, first we run ecldoc using subprocess in python to obtain original xml output from ecldoc -M Then we use LXML to parse this XMLOriginal to generate new XML. New XML contains : Signature of each attribute by reading corresponding source code . Documentation parser was rewritten in python.

In first week, we completed following topics : Modification in HPCC Platform source code to implement : Function like objects Meta information Inheritance Information Information about whether the object is inherited or overriden. These Modification were used to identify the necessary information for documentation. Basic Type System Information was developed (but put on shelf for now). Meta Tree was linked with source code to identify the attribute signature. Imports were linked with appropriate file. Attribute fullnames were appended to find actual source (internal or external).

I will be joining officially from tomorrow (29th May).  1) Current implementation of meta (with additions I made) works on records, modules, interfaces, function like objects(functions, transform, embed, module with params). 2) Type information is generated using getFriendlyTypeStr for the time being. As gavin replied, we would need to replace that more informative system. 3) Module Inheritance works by including all inherited attributes in child module automatically. If inherited attribute is overriden, new position info is generated otherwise position info in parent in maintained. 4) Record Inheritance works similarly by including all inherited fields in new child record. Currently, I am using ECLIR as a template to understand how the expression tree is to be navigated and how to access various information regarding objects in an ecl file since ECLIR provides a lot of similar info that we need for meta, albeit in a different format. In first week, I will work on f