Map Parsing

[awilkins]

Vast majority of (wrong) lints in our codebase come from map rules ... so starting some map parsing

Map rules start

RULEVAR RULENAMEIDENTIFIER

And then

{
INCDIR  DDMNAME   FIELDNAME   ;
|
INCDIC FREERULENAME          ;
|
INCDIC                    ;
}

The RULEVAR continues until another RULEVAR starts, or a normal object end happens.

• INCDIR means "use the Predict rule for this Adabas field"
  • These rules get added to views of Adabas files in maps automatically
  • These are possibly what the docs call "Predict automatic rules"
  • Not sure if this inlines the code in the compiled object or calls out to some kind of hook
  • The (only)? check possible here is that the field exists in the named DDM
• INCDIR with a name is a Predict Free Rule
  • These make the compiler inline a chunk of code stored in a Predict Verification in your FDIC database
  • Without having a local copy of the code, doing any work with these is awkward
• INCDIR with no name is an inline rule
  • These are the source of the trouble

Inline rules are effectively a nested program. Inline rules are allowed their own DEFINE DATA block.

Inline rules may access all the variables in the scope of the current map plus any variables declared in their own block.

---

Where this branch is right now ....

• Added DDM and field, properties for the INCDIR nodes
• Made INCDIC nodes a node-with-body
• Added a define block parser to INCDIC nodes
• Added a means for variable resolution to find variables in the current scoped DEFINE block in rules

What needs to be added

• Hover no longer works (nor "goto def" etc), the nodes in the rule programs aren't working right.

[awilkins]

OK, so failing the consumeIncDir test case which checks that this parses

INCDIR #VAR #VAR1 #VAR2;

In our codebase this has hits (and all INCDIR statements take this form)

(INCDIR followed by two regions of non-whitespace before semicolon)

INCDIR\s+(\S+)\s+(\S+)\s+;

There are no hits for the pattern in the test

(INCDIR followed by three regions of non-whitespace before semicolon)

INCDIC\s+(\S+)\s+(\S+)\s+(\S+)\s*;

Does your code have any hits on this regex @markusamshove ?

[awilkins]

This seems to be some kind of indirection syntax

[markusamshove]

I am impressed that you found some kind of documentation to explain what those things even do/mean. I've tried to understand them for some time now, because we also had a lot of those errors and I'm trying to build a map editor.

We do have maps with INCDIR, INCDIC and RULEVAR.

We do not have any matches on this regex:

[awilkins]

There are some basic docs about rules here, but they're not very good at explaining the source code structure. Happily I have lots of examples to work with ....

Ultimately I really want to get rid of all the Predict Free Rules ... or at least convert them into inline rules in a pre-build step, rather than depend on source code which is hidden away in a database somewhere and isn't under version control.

---

Aha, there's something about that & syntax ....

An ampersand (&) within the source code of a processing rule is dynamically replaced by the fully-qualified name of the field for which the rule is defined.

So

MARK *&(L-ERROR-FIELD)

Applies to a field array and marks the L-ERROR-FIELDth element....

---

Most of the error marks are gone from my maps now (that error on & remains) but hover isn't working, like the syntax nodes are disconnected or something. I've created a "nested" define class that allows looking up variables in both define blocks in scope, but I haven't implemented all the members.

---

Map Editor

The N1 map editor stores a bunch of metadata in magic comments but it's entirely possible to just write maps as source.

Sadly when you do, sometimes the map editor will completely ruin your source code if it doesn't "understand" it... even if it catalogs and works just fine.

[sonarqubecloud]

Quality Gate failed

Failed conditions
28.2% Coverage on New Code (required ≥ 80%)
B Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

[awilkins]

At this point, codebase (which mostly catalogs fine but contains errors that are more obvious with from less noisy linting .. ) has gone from

	Lints	Errors
Before	49,542	38,482
After	11,922	851

Next biggest categories of diagnostics that don't look like errors in the code...

---

• Label qualifiers

HIST. HISTOGRAM MY-HISTOGRAM FOR MY-TABLE
    STARTING FROM L-KEY-VALUE

  IF HIST.#RECORD-KEY   /* "unresolved reference" ; labels are a valid qualifier to refer to loop views

END-HISTOGRAM

---

• More label qualifiers ( (ref) parenthesis style)

This seems to be valid in most places you can accept a database field as an operand ...

LB1. READ MY-TABLE BY THE-KEY STARTING FROM 'THE TOP'

  IF AMBIGUOUS-FIELD (LB1.) = 'A VALUE' /*  NPP029, ambiguous and needs to be qualified
    IGNORE
  END-IF

END-READ

---

• Unresolved external module

For modules that are known to be present but not visible in the source code.

Would be nice to have a way to declare external module names (extending the assumptions like USRnnnnN is a system API)

---

• "Identifier expected, but got ... (keyword)"

this revision makes a stab at widening the pool of keywords that can be identifiers ... but we have a lot more and this method starts to break tests if you include them all. Need a better way to do this.

---