Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

Frameworking with Dyalog OO-APL

Tobias Wessels

2021

APL user meeting
Tobias Wessels, 22.11.2021

Table of Contents

Different frameworks/techniques used: Application templates, ActiveRecord, Xsd2Code, DUnit
APL user meeting
Tobias Wessels, 22.11.2021

Outline

Goal

Develop a webstore backend for easily managing price calculations of a web store.

Requirements

  • Persistent data storage for products, prices and promotions in our webstore
  • An exchange file format for saving, exchanging and loading customer data (eg. personal information, current state of the shopping cart)
  • Business logic for price calculations
APL user meeting
Tobias Wessels, 22.11.2021

Programming Techniques

  • Object-relational-modelling & ActiveRecord design pattern
  • Model-definitions using xsd (xml schema definition) files
  • Metaprogramming & domain-specific languages
  • Test-driven-development
  • Techniques, that are common in other programming languages and frameworks (eg. RubyOnRails, Django)
  • Little prior experience in programming professionally
APL user meeting
Tobias Wessels, 22.11.2021

Project setup

  • A package repository is used to pull in "third party" dependencies.
  • For code management, a version control system is used to track changes and manage the development cycle.
  • New releases of packages are uploaded to the central package repository as needed.
  • For an easy and standardized setup, a program called Bootstrapping is used.
APL user meeting
Tobias Wessels, 22.11.2021

Package repository

Utils Xsd2Code Dyadox Bootstrapping Webstore DUnit Package Repository - Loader class - Startup script
  • Multiple packages use common functionality. They all depend on Utils.
  • Some packages pull in other packages for additional features, eg. Xsd2Code.
  • Some tools need to load other programs to programmatically interact with them, eg. DUnit or Dyadox.
  • Sometimes, different versions of the same package must be loaded at the same time.
Utils Webstore Working directory
Bootstrapping
Tobias Wessels, 22.11.2021

Setting up an "empty" application

  • Create a default folder structure for your project.
  • Copy the latest Startup script into your project directory.
  • Provide a scripting facility to enable build scripts.
  • Provide a framework for pulling in additional dependencies.
  • Setup basic dependencies useful for any project: (Utils,) DUnit, Dyadox
  • Create a template for an empty application class.
Bootstrapping
Tobias Wessels, 22.11.2021

Running bootstrapping


		
Bootstrapping
Tobias Wessels, 22.11.2021

App skeleton

Folder structure of a newly created app.
  • Startup+StoreStartup: Startup script + hooks
  • src: source code of our app
  • tests: DUnit test cases
  • scripts: build scripts
  • loader_konfiguration.json: configuration file specifying dependencies and their versions
Bootstrapping
Tobias Wessels, 22.11.2021

Standard "binaries"

Starters: Store, DUnit, Dyadox
Bootstrapping
Tobias Wessels, 22.11.2021

Loader configuration

			
		
APL user meeting
Tobias Wessels, 22.11.2021

Development overview

  1. Persistent storage of product information:
    1. modelling of products
    2. create data structures
    3. create database
    4. create api to load data from the database
  2. Exchange format for user data:
    1. modelling of user data
    2. create data structures
    3. create api to import/export user data to the filesystem
  3. Implement business logic:
    1. implement price calculations
    2. write automated tests for the implementation
APL user meeting
Tobias Wessels, 22.11.2021

Persistent storage of product information

  1. Product data is persistent data, loadable on each startup of the application.
  2. Product data will be stored across multiple entities in a single database.
  3. Entities will be directly mapped onto classes in our program.
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Object-relational mapping (ORM)

  • A technique to query and manipulate data from a database using an object-oriented approach.
  • Specify data model in a domain-specific language.
  • Provide an entry point for querying and caching the database.
  • Generate data-structures (APL code) that can be used to access the data.
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Procedure: Setting up our ORM

  1. Write down specification of the database layout in OrmEntity classes
  2. Create a config file for the data access layer (a subclass of OrmConnectionBasisklasse)
  3. Setup database via SQL scripts and access database from APL
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Database schema

Database schema
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Specifying our model: OrmProduct


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Specifying our model: OrmPromotion


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Specifying our model: EnumPromoType


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Setting up the data access layer


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Accessing the ORM

Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Generated sql scripts: CREATE


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Generated sql scripts: PRIMARY KEYS


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

Generated sql scripts: FOREIGN KEYS


		
Persistent storage of product information
APL user meeting
Tobias Wessels, 22.11.2021

(Semi-generated) sql scripts: INSERTS


		
APL user meeting
Tobias Wessels, 22.11.2021
APL user meeting
Tobias Wessels, 22.11.2021

Exchange format for user data

  1. User input (eg. customer name, address, shopping cart) is volatile data.
  2. Users should be able to save and restore their inputs.
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Xsd2Code is a framework for ...

  • modelling user data
  • creating data structures (auto-generated classes)
  • creating an api to import/export user data to the filesystem (configurable existing code)

For steps 2 and 3 we only need to create a configuration file to customize existing classes.

Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Data structure requirements

Compositional structure with three classes: customer, cart and item

Ideally, the data structure will support following features:

  1. A distinction between mutable and immutable fields
  2. Composition of data structures
  3. Optionally, reverse "pointers" to owner classes in a composition structure
  4. Possibility to subclass nodes in the structure to work with the data (write api methods)
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Procedure: Setting up Xsd2Code

  1. Write an XML Schema Definition file to specify the data model.
  2. Write a configuration file for the Xsd2Code framework.
  3. Generate classes and get familiar with their api.
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Writing an XSD file

  • Classes are represented by complex types.
  • Fields are either elements (=mutable) or attributes (=immutable).
  • Attributes must be specified at construction time, ie. in the constructor. "required" means: they must be passed as arguments.
  • Specify multiplicity with "minOccurs" and "maxOccurs". Field names will be pluralized (as required).
  • Notice ownership-flag in the documentation nodes!
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Writing an XSD file: examples


		
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Configure the Xsd2Code framework


		
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Code generation

Add one line to loader_configuration.json:

"Xsd2Code": "${repo}/Xsd2Code/tags/v1.05",

Run code generation:


		
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Generated files

Four generated files: Customer, Cart, Items and CustomerFactory
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Using the xsd2code api


		
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Exported XML file of a customer


		
APL user meeting
Tobias Wessels, 22.11.2021

Implementing business logic

  1. Aim: Write code that calculates prices of our shopping cart.
  2. Business logic closely related to our data models should be implemented in the respective classes.
  3. Readability of code is important! Separate technical, repetitive code from business logic.
  4. Should be well tested to avoid releasing bugs into productive code.
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Procedure: Adding price calculations

  1. Extend/subclass Cart and Item from our customer model.
  2. Implement business logic using the object oriented programming paradigm.
  3. Using the DUnit framework and toolset, write automated tests to check the implementation.
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Subclassing the customer model


		
Exchange format for user data
APL user meeting
Tobias Wessels, 22.11.2021

Generated files - subclasses

Four generated files: Customer, BCart, BItems and CustomerFactory
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Adding business logic to the customer model


		
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Familiarize with the new API


		
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Testing the implementation


		
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Running DUnit


		
Implementing business logic
APL user meeting
Tobias Wessels, 22.11.2021

Debugging Errors and Failures

Dyalog window with debugger at failed test.
Small steps lead to great advancements
APL user meeting
Tobias Wessels, 22.11.2021

Recap: What was achieved

  1. Setting up the application architecture.Execute Bootstrapping
  2. Specifying entities in our database and mapping these to (auto-generated) objects in our program.Write ORM classes + config
  3. Specifying data representation of volatile objects, auto-generated data structures and XML import/export.Write XSD spec + config
  4. Implementing business logic in the appropriate classes and ensuring correctness of the code.Write code + tests
APL user meeting
Tobias Wessels, 22.11.2021

AddOns

Documenting code
Tobias Wessels, 22.11.2021

Read the source ...

... and if it's not self explanatory, hopefully at least there is useful documentation. Dyadox may help you keeping your code well documented:

  1. Keep your documentation (simple APL comments) closely linked with your source.
    • Use (extended) markdown syntax.
    • Embed LaTeX formulas into your docstrings.
  2. Enhance readability and navigate your documentation freely in HTML format.
Documenting code
Tobias Wessels, 22.11.2021

Writing useful comments


		
Documenting code
Tobias Wessels, 22.11.2021

Reading documentation

Dyadox html documentation