forked from Informatica-EIC/REST-API-Samples
-
Notifications
You must be signed in to change notification settings - Fork 2
Expand file tree
/
Copy pathsrc_externalDBLinker_py.html
More file actions
496 lines (496 loc) · 108 KB
/
Copy pathsrc_externalDBLinker_py.html
File metadata and controls
496 lines (496 loc) · 108 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=emulateIE7" />
<title>Coverage for src\externalDBLinker.py: 0%</title>
<link rel="stylesheet" href="style.css" type="text/css">
<script type="text/javascript" src="jquery.min.js"></script>
<script type="text/javascript" src="jquery.hotkeys.js"></script>
<script type="text/javascript" src="jquery.isonscreen.js"></script>
<script type="text/javascript" src="coverage_html.js"></script>
<script type="text/javascript">
jQuery(document).ready(coverage.pyfile_ready);
</script>
</head>
<body class="pyfile">
<div id="header">
<div class="content">
<h1>Coverage for <b>src\externalDBLinker.py</b> :
<span class="pc_cov">0%</span>
</h1>
<img id="keyboard_icon" src="keybd_closed.png" alt="Show keyboard shortcuts" />
<h2 class="stats">
157 statements
<button type="button" class="run shortkey_r button_toggle_run" title="Toggle lines run">0 run</button>
<button type="button" class="mis show_mis shortkey_m button_toggle_mis" title="Toggle lines missing">157 missing</button>
<button type="button" class="exc show_exc shortkey_x button_toggle_exc" title="Toggle lines excluded">0 excluded</button>
</h2>
</div>
</div>
<div class="help_panel">
<img id="panel_icon" src="keybd_open.png" alt="Hide keyboard shortcuts" />
<p class="legend">Hot-keys on this page</p>
<div>
<p class="keyhelp">
<span class="key">r</span>
<span class="key">m</span>
<span class="key">x</span>
<span class="key">p</span> toggle line displays
</p>
<p class="keyhelp">
<span class="key">j</span>
<span class="key">k</span> next/prev highlighted chunk
</p>
<p class="keyhelp">
<span class="key">0</span> (zero) top of page
</p>
<p class="keyhelp">
<span class="key">1</span> (one) first highlighted chunk
</p>
</div>
</div>
<div id="source">
<p id="t1" class="pln"><span class="n"><a href="#t1">1</a></span><span class="t"><span class="str">"""</span> </span><span class="r"></span></p>
<p id="t2" class="pln"><span class="n"><a href="#t2">2</a></span><span class="t"><span class="str">Created on Aug 7, 2018</span> </span><span class="r"></span></p>
<p id="t3" class="pln"><span class="n"><a href="#t3">3</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t4" class="pln"><span class="n"><a href="#t4">4</a></span><span class="t"><span class="str">@author: dwrigley</span> </span><span class="r"></span></p>
<p id="t5" class="pln"><span class="n"><a href="#t5">5</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t6" class="pln"><span class="n"><a href="#t6">6</a></span><span class="t"><span class="str">resolve links for External Database/Schema objects</span> </span><span class="r"></span></p>
<p id="t7" class="pln"><span class="n"><a href="#t7">7</a></span><span class="t"><span class="str">write a custom lineage file to import the links</span> </span><span class="r"></span></p>
<p id="t8" class="pln"><span class="n"><a href="#t8">8</a></span><span class="t"><span class="str">or</span> </span><span class="r"></span></p>
<p id="t9" class="pln"><span class="n"><a href="#t9">9</a></span><span class="t"><span class="str">options include to create links directly (via API) - via -i switch (or --edcImport)</span> </span><span class="r"></span></p>
<p id="t10" class="pln"><span class="n"><a href="#t10">10</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t11" class="pln"><span class="n"><a href="#t11">11</a></span><span class="t"><span class="str">"""</span> </span><span class="r"></span></p>
<p id="t12" class="mis show_mis"><span class="n"><a href="#t12">12</a></span><span class="t"><span class="key">import</span> <span class="nam">json</span> </span><span class="r"></span></p>
<p id="t13" class="mis show_mis"><span class="n"><a href="#t13">13</a></span><span class="t"><span class="key">import</span> <span class="nam">csv</span> </span><span class="r"></span></p>
<p id="t14" class="mis show_mis"><span class="n"><a href="#t14">14</a></span><span class="t"><span class="key">from</span> <span class="nam">src</span><span class="op">.</span><span class="nam">edc_utilities</span> <span class="key">import</span> <span class="nam">edcSessionHelper</span><span class="op">,</span> <span class="nam">edcutils</span> </span><span class="r"></span></p>
<p id="t15" class="mis show_mis"><span class="n"><a href="#t15">15</a></span><span class="t"><span class="key">import</span> <span class="nam">time</span> </span><span class="r"></span></p>
<p id="t16" class="mis show_mis"><span class="n"><a href="#t16">16</a></span><span class="t"><span class="key">import</span> <span class="nam">sys</span> </span><span class="r"></span></p>
<p id="t17" class="mis show_mis"><span class="n"><a href="#t17">17</a></span><span class="t"><span class="key">import</span> <span class="nam">argparse</span> </span><span class="r"></span></p>
<p id="t18" class="mis show_mis"><span class="n"><a href="#t18">18</a></span><span class="t"><span class="key">import</span> <span class="nam">urllib3</span> </span><span class="r"></span></p>
<p id="t19" class="mis show_mis"><span class="n"><a href="#t19">19</a></span><span class="t"><span class="key">import</span> <span class="nam">os</span> </span><span class="r"></span></p>
<p id="t20" class="pln"><span class="n"><a href="#t20">20</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t21" class="mis show_mis"><span class="n"><a href="#t21">21</a></span><span class="t"><span class="nam">urllib3</span><span class="op">.</span><span class="nam">disable_warnings</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t22" class="pln"><span class="n"><a href="#t22">22</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t23" class="pln"><span class="n"><a href="#t23">23</a></span><span class="t"><span class="com"># set edc helper session + variables (easy/re-useable connection to edc api)</span> </span><span class="r"></span></p>
<p id="t24" class="mis show_mis"><span class="n"><a href="#t24">24</a></span><span class="t"><span class="nam">edcHelper</span> <span class="op">=</span> <span class="nam">edcSessionHelper</span><span class="op">.</span><span class="nam">EDCSession</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t25" class="pln"><span class="n"><a href="#t25">25</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t26" class="pln"><span class="n"><a href="#t26">26</a></span><span class="t"><span class="com"># define script command-line parameters (in global scope for gooey/wooey)</span> </span><span class="r"></span></p>
<p id="t27" class="mis show_mis"><span class="n"><a href="#t27">27</a></span><span class="t"><span class="nam">parser</span> <span class="op">=</span> <span class="nam">argparse</span><span class="op">.</span><span class="nam">ArgumentParser</span><span class="op">(</span><span class="nam">parents</span><span class="op">=</span><span class="op">[</span><span class="nam">edcHelper</span><span class="op">.</span><span class="nam">argparser</span><span class="op">]</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t28" class="pln"><span class="n"><a href="#t28">28</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t29" class="pln"><span class="n"><a href="#t29">29</a></span><span class="t"><span class="com"># add args specific to this utility (left/right resource, schema, classtype...)</span> </span><span class="r"></span></p>
<p id="t30" class="mis show_mis"><span class="n"><a href="#t30">30</a></span><span class="t"><span class="nam">parser</span><span class="op">.</span><span class="nam">add_argument</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t31" class="pln"><span class="n"><a href="#t31">31</a></span><span class="t"> <span class="str">"-f"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t32" class="pln"><span class="n"><a href="#t32">32</a></span><span class="t"> <span class="str">"--csv_file_name"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t33" class="pln"><span class="n"><a href="#t33">33</a></span><span class="t"> <span class="nam">default</span><span class="op">=</span><span class="str">"dbms_externalDBLinks.csv"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t34" class="pln"><span class="n"><a href="#t34">34</a></span><span class="t"> <span class="nam">required</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t35" class="pln"><span class="n"><a href="#t35">35</a></span><span class="t"> <span class="nam">help</span><span class="op">=</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t36" class="pln"><span class="n"><a href="#t36">36</a></span><span class="t"> <span class="str">"csv file to create/write (no folder) default=dbms_externalDBLinks.csv "</span> </span><span class="r"></span></p>
<p id="t37" class="pln"><span class="n"><a href="#t37">37</a></span><span class="t"> <span class="op">)</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t38" class="pln"><span class="n"><a href="#t38">38</a></span><span class="t"><span class="op">)</span> </span><span class="r"></span></p>
<p id="t39" class="mis show_mis"><span class="n"><a href="#t39">39</a></span><span class="t"><span class="nam">parser</span><span class="op">.</span><span class="nam">add_argument</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t40" class="pln"><span class="n"><a href="#t40">40</a></span><span class="t"> <span class="str">"-o"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t41" class="pln"><span class="n"><a href="#t41">41</a></span><span class="t"> <span class="str">"--outDir"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t42" class="pln"><span class="n"><a href="#t42">42</a></span><span class="t"> <span class="nam">default</span><span class="op">=</span><span class="str">"out"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t43" class="pln"><span class="n"><a href="#t43">43</a></span><span class="t"> <span class="nam">required</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t44" class="pln"><span class="n"><a href="#t44">44</a></span><span class="t"> <span class="nam">help</span><span class="op">=</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t45" class="pln"><span class="n"><a href="#t45">45</a></span><span class="t"> <span class="str">"output folder to write results - default = ./out "</span> </span><span class="r"></span></p>
<p id="t46" class="pln"><span class="n"><a href="#t46">46</a></span><span class="t"> <span class="str">" - will create folder if it does not exist"</span> </span><span class="r"></span></p>
<p id="t47" class="pln"><span class="n"><a href="#t47">47</a></span><span class="t"> <span class="op">)</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t48" class="pln"><span class="n"><a href="#t48">48</a></span><span class="t"><span class="op">)</span> </span><span class="r"></span></p>
<p id="t49" class="pln"><span class="n"><a href="#t49">49</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t50" class="mis show_mis"><span class="n"><a href="#t50">50</a></span><span class="t"><span class="nam">parser</span><span class="op">.</span><span class="nam">add_argument</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t51" class="pln"><span class="n"><a href="#t51">51</a></span><span class="t"> <span class="str">"-i"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t52" class="pln"><span class="n"><a href="#t52">52</a></span><span class="t"> <span class="str">"--edcimport"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t53" class="pln"><span class="n"><a href="#t53">53</a></span><span class="t"> <span class="nam">default</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t54" class="pln"><span class="n"><a href="#t54">54</a></span><span class="t"> <span class="com"># type=bool,</span> </span><span class="r"></span></p>
<p id="t55" class="pln"><span class="n"><a href="#t55">55</a></span><span class="t"> <span class="nam">action</span><span class="op">=</span><span class="str">'store_true'</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t56" class="pln"><span class="n"><a href="#t56">56</a></span><span class="t"> <span class="nam">help</span><span class="op">=</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t57" class="pln"><span class="n"><a href="#t57">57</a></span><span class="t"> <span class="str">"use the rest api to create the custom lineage resource and start the import process "</span> </span><span class="r"></span></p>
<p id="t58" class="pln"><span class="n"><a href="#t58">58</a></span><span class="t"> <span class="op">)</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t59" class="pln"><span class="n"><a href="#t59">59</a></span><span class="t"><span class="op">)</span> </span><span class="r"></span></p>
<p id="t60" class="pln"><span class="n"><a href="#t60">60</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t61" class="mis show_mis"><span class="n"><a href="#t61">61</a></span><span class="t"><span class="nam">parser</span><span class="op">.</span><span class="nam">add_argument</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t62" class="pln"><span class="n"><a href="#t62">62</a></span><span class="t"> <span class="str">"-rn"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t63" class="pln"><span class="n"><a href="#t63">63</a></span><span class="t"> <span class="str">"--lineageResourceName"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t64" class="pln"><span class="n"><a href="#t64">64</a></span><span class="t"> <span class="nam">default</span><span class="op">=</span><span class="str">"externalDBlinker_lineage"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t65" class="pln"><span class="n"><a href="#t65">65</a></span><span class="t"> <span class="nam">required</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t66" class="pln"><span class="n"><a href="#t66">66</a></span><span class="t"> <span class="nam">help</span><span class="op">=</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t67" class="pln"><span class="n"><a href="#t67">67</a></span><span class="t"> <span class="str">"custom lineage resource name to create/update - default value=externalDBlinker_lineage"</span> </span><span class="r"></span></p>
<p id="t68" class="pln"><span class="n"><a href="#t68">68</a></span><span class="t"> <span class="op">)</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t69" class="pln"><span class="n"><a href="#t69">69</a></span><span class="t"><span class="op">)</span> </span><span class="r"></span></p>
<p id="t70" class="mis show_mis"><span class="n"><a href="#t70">70</a></span><span class="t"><span class="nam">parser</span><span class="op">.</span><span class="nam">add_argument</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t71" class="pln"><span class="n"><a href="#t71">71</a></span><span class="t"> <span class="str">"-rt"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t72" class="pln"><span class="n"><a href="#t72">72</a></span><span class="t"> <span class="str">"--lineageResourceTemplate"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t73" class="pln"><span class="n"><a href="#t73">73</a></span><span class="t"> <span class="nam">default</span><span class="op">=</span><span class="str">"template/custom_lineage_template.json"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t74" class="pln"><span class="n"><a href="#t74">74</a></span><span class="t"> <span class="nam">required</span><span class="op">=</span><span class="key">False</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t75" class="pln"><span class="n"><a href="#t75">75</a></span><span class="t"> <span class="nam">help</span><span class="op">=</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t76" class="pln"><span class="n"><a href="#t76">76</a></span><span class="t"> <span class="str">"custom lineage resource template (json format) use for creating a new resource - default value=template/custom_lineage_template.json"</span> </span><span class="r"></span></p>
<p id="t77" class="pln"><span class="n"><a href="#t77">77</a></span><span class="t"> <span class="op">)</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t78" class="pln"><span class="n"><a href="#t78">78</a></span><span class="t"><span class="op">)</span> </span><span class="r"></span></p>
<p id="t79" class="pln"><span class="n"><a href="#t79">79</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t80" class="pln"><span class="n"><a href="#t80">80</a></span><span class="t"><span class="com"># waitToComplete setting is not active (and probably not useful anyway)</span> </span><span class="r"></span></p>
<p id="t81" class="mis show_mis"><span class="n"><a href="#t81">81</a></span><span class="t"><span class="nam">waitToComplete</span> <span class="op">=</span> <span class="key">False</span> </span><span class="r"></span></p>
<p id="t82" class="pln"><span class="n"><a href="#t82">82</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t83" class="pln"><span class="n"><a href="#t83">83</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t84" class="mis show_mis"><span class="n"><a href="#t84">84</a></span><span class="t"><span class="key">def</span> <span class="nam">getColumnsForTable</span><span class="op">(</span><span class="nam">tableId</span><span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t85" class="pln"><span class="n"><a href="#t85">85</a></span><span class="t"> <span class="str">"""</span> </span><span class="r"></span></p>
<p id="t86" class="pln"><span class="n"><a href="#t86">86</a></span><span class="t"><span class="str"> get all columns for a table and return in a dict key=column name</span> </span><span class="r"></span></p>
<p id="t87" class="pln"><span class="n"><a href="#t87">87</a></span><span class="t"><span class="str"> (lowercase) val=column id</span> </span><span class="r"></span></p>
<p id="t88" class="pln"><span class="n"><a href="#t88">88</a></span><span class="t"><span class="str"> use the GET /access/2/catalog/data/objects id=tableId</span> </span><span class="r"></span></p>
<p id="t89" class="pln"><span class="n"><a href="#t89">89</a></span><span class="t"><span class="str"> look for columns via com.infa.ldm.relational.TableColumn</span> </span><span class="r"></span></p>
<p id="t90" class="pln"><span class="n"><a href="#t90">90</a></span><span class="t"><span class="str"> or com.infa.ldm.relational.ViewViewColumn relationship type</span> </span><span class="r"></span></p>
<p id="t91" class="pln"><span class="n"><a href="#t91">91</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t92" class="pln"><span class="n"><a href="#t92">92</a></span><span class="t"><span class="str"> """</span> </span><span class="r"></span></p>
<p id="t93" class="pln"><span class="n"><a href="#t93">93</a></span><span class="t"> <span class="com"># TODO: this could be moved to edcutils.py?</span> </span><span class="r"></span></p>
<p id="t94" class="pln"><span class="n"><a href="#t94">94</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t95" class="mis show_mis"><span class="n"><a href="#t95">95</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\t\tget cols for table:"</span> <span class="op">+</span> <span class="nam">tableId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t96" class="mis show_mis"><span class="n"><a href="#t96">96</a></span><span class="t"> <span class="nam">parameters</span> <span class="op">=</span> <span class="op">{</span> </span><span class="r"></span></p>
<p id="t97" class="pln"><span class="n"><a href="#t97">97</a></span><span class="t"> <span class="str">"id"</span><span class="op">:</span> <span class="nam">tableId</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t98" class="pln"><span class="n"><a href="#t98">98</a></span><span class="t"> <span class="str">"offset"</span><span class="op">:</span> <span class="num">0</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t99" class="pln"><span class="n"><a href="#t99">99</a></span><span class="t"> <span class="str">"pageSize"</span><span class="op">:</span> <span class="num">1</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t100" class="pln"><span class="n"><a href="#t100">100</a></span><span class="t"> <span class="op">}</span> <span class="com"># pagesize can be 1 - since we are only passing the id</span> </span><span class="r"></span></p>
<p id="t101" class="mis show_mis"><span class="n"><a href="#t101">101</a></span><span class="t"> <span class="nam">colResp</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">session</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t102" class="pln"><span class="n"><a href="#t102">102</a></span><span class="t"> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">baseUrl</span> <span class="op">+</span> <span class="str">"/access/2/catalog/data/objects"</span><span class="op">,</span> <span class="nam">params</span><span class="op">=</span><span class="nam">parameters</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t103" class="pln"><span class="n"><a href="#t103">103</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t104" class="mis show_mis"><span class="n"><a href="#t104">104</a></span><span class="t"> <span class="nam">tableJson</span> <span class="op">=</span> <span class="nam">colResp</span><span class="op">.</span><span class="nam">json</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t105" class="pln"><span class="n"><a href="#t105">105</a></span><span class="t"> <span class="com"># print(colResp.status_code)</span> </span><span class="r"></span></p>
<p id="t106" class="mis show_mis"><span class="n"><a href="#t106">106</a></span><span class="t"> <span class="nam">colIds</span> <span class="op">=</span> <span class="nam">dict</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t107" class="mis show_mis"><span class="n"><a href="#t107">107</a></span><span class="t"> <span class="key">for</span> <span class="nam">extDBItem</span> <span class="key">in</span> <span class="nam">tableJson</span><span class="op">[</span><span class="str">"items"</span><span class="op">]</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t108" class="pln"><span class="n"><a href="#t108">108</a></span><span class="t"> <span class="com"># print(">>>")</span> </span><span class="r"></span></p>
<p id="t109" class="mis show_mis"><span class="n"><a href="#t109">109</a></span><span class="t"> <span class="key">for</span> <span class="nam">dst</span> <span class="key">in</span> <span class="nam">extDBItem</span><span class="op">[</span><span class="str">"dstLinks"</span><span class="op">]</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t110" class="mis show_mis"><span class="n"><a href="#t110">110</a></span><span class="t"> <span class="nam">assoc</span> <span class="op">=</span> <span class="nam">dst</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"association"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t111" class="mis show_mis"><span class="n"><a href="#t111">111</a></span><span class="t"> <span class="nam">colId</span> <span class="op">=</span> <span class="nam">dst</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"id"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t112" class="mis show_mis"><span class="n"><a href="#t112">112</a></span><span class="t"> <span class="nam">colName</span> <span class="op">=</span> <span class="nam">dst</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"name"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t113" class="pln"><span class="n"><a href="#t113">113</a></span><span class="t"> <span class="com"># print("\tdst..." + assoc)</span> </span><span class="r"></span></p>
<p id="t114" class="mis show_mis"><span class="n"><a href="#t114">114</a></span><span class="t"> <span class="key">if</span> <span class="nam">assoc</span> <span class="key">in</span> <span class="op">(</span> </span><span class="r"></span></p>
<p id="t115" class="pln"><span class="n"><a href="#t115">115</a></span><span class="t"> <span class="str">"com.infa.ldm.relational.TableColumn"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t116" class="pln"><span class="n"><a href="#t116">116</a></span><span class="t"> <span class="str">"com.infa.ldm.relational.ViewViewColumn"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t117" class="pln"><span class="n"><a href="#t117">117</a></span><span class="t"> <span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t118" class="pln"><span class="n"><a href="#t118">118</a></span><span class="t"> <span class="com"># key = lower_case name - for case-insensitive lookup</span> </span><span class="r"></span></p>
<p id="t119" class="mis show_mis"><span class="n"><a href="#t119">119</a></span><span class="t"> <span class="nam">colIds</span><span class="op">[</span><span class="nam">colName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span><span class="op">]</span> <span class="op">=</span> <span class="nam">colId</span> </span><span class="r"></span></p>
<p id="t120" class="pln"><span class="n"><a href="#t120">120</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t121" class="mis show_mis"><span class="n"><a href="#t121">121</a></span><span class="t"> <span class="key">return</span> <span class="nam">colIds</span> </span><span class="r"></span></p>
<p id="t122" class="pln"><span class="n"><a href="#t122">122</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t123" class="pln"><span class="n"><a href="#t123">123</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t124" class="mis show_mis"><span class="n"><a href="#t124">124</a></span><span class="t"><span class="key">def</span> <span class="nam">processExternalDB</span><span class="op">(</span><span class="nam">dbId</span><span class="op">,</span> <span class="nam">classType</span><span class="op">,</span> <span class="nam">dbName</span><span class="op">,</span> <span class="nam">resType</span><span class="op">,</span> <span class="nam">resName</span><span class="op">,</span> <span class="nam">colWriter</span><span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t125" class="pln"><span class="n"><a href="#t125">125</a></span><span class="t"> <span class="str">"""</span> </span><span class="r"></span></p>
<p id="t126" class="pln"><span class="n"><a href="#t126">126</a></span><span class="t"><span class="str"> dbId = the id of the external database object</span> </span><span class="r"></span></p>
<p id="t127" class="pln"><span class="n"><a href="#t127">127</a></span><span class="t"><span class="str"> classType = the classtype of the external database oject</span> </span><span class="r"></span></p>
<p id="t128" class="pln"><span class="n"><a href="#t128">128</a></span><span class="t"><span class="str"> dbNAme - the name of the external db object</span> </span><span class="r"></span></p>
<p id="t129" class="pln"><span class="n"><a href="#t129">129</a></span><span class="t"><span class="str"> resType - the resource type e.g Oracle, SQLServer that created the</span> </span><span class="r"></span></p>
<p id="t130" class="pln"><span class="n"><a href="#t130">130</a></span><span class="t"><span class="str"> external db object</span> </span><span class="r"></span></p>
<p id="t131" class="pln"><span class="n"><a href="#t131">131</a></span><span class="t"><span class="str"> resName - the resource name that created the external db object</span> </span><span class="r"></span></p>
<p id="t132" class="pln"><span class="n"><a href="#t132">132</a></span><span class="t"><span class="str"> colWriter - the csv File object to write any lineage entries</span> </span><span class="r"></span></p>
<p id="t133" class="pln"><span class="n"><a href="#t133">133</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t134" class="pln"><span class="n"><a href="#t134">134</a></span><span class="t"><span class="str"> process outline:-</span> </span><span class="r"></span></p>
<p id="t135" class="pln"><span class="n"><a href="#t135">135</a></span><span class="t"><span class="str"> get lineage to the table level (2 parent/child hops) from the external database</span> </span><span class="r"></span></p>
<p id="t136" class="pln"><span class="n"><a href="#t136">136</a></span><span class="t"><span class="str"> for each table, find the actual table that should be linked (case insensitive)</span> </span><span class="r"></span></p>
<p id="t137" class="pln"><span class="n"><a href="#t137">137</a></span><span class="t"><span class="str"> if only 1 table was found (otherwise - print an error message)</span> </span><span class="r"></span></p>
<p id="t138" class="pln"><span class="n"><a href="#t138">138</a></span><span class="t"><span class="str"> link the table and all columns</span> </span><span class="r"></span></p>
<p id="t139" class="pln"><span class="n"><a href="#t139">139</a></span><span class="t"><span class="str"> """</span> </span><span class="r"></span></p>
<p id="t140" class="pln"><span class="n"><a href="#t140">140</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t141" class="mis show_mis"><span class="n"><a href="#t141">141</a></span><span class="t"> <span class="nam">dbUnknown</span> <span class="op">=</span> <span class="key">False</span> </span><span class="r"></span></p>
<p id="t142" class="pln"><span class="n"><a href="#t142">142</a></span><span class="t"> <span class="com"># counters for # of links created</span> </span><span class="r"></span></p>
<p id="t143" class="mis show_mis"><span class="n"><a href="#t143">143</a></span><span class="t"> <span class="nam">tabLinks</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t144" class="mis show_mis"><span class="n"><a href="#t144">144</a></span><span class="t"> <span class="nam">colLinks</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t145" class="mis show_mis"><span class="n"><a href="#t145">145</a></span><span class="t"> <span class="nam">errors</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t146" class="pln"><span class="n"><a href="#t146">146</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t147" class="pln"><span class="n"><a href="#t147">147</a></span><span class="t"> <span class="com"># 'External' can be used if the database name is not known (sqlserver use case)</span> </span><span class="r"></span></p>
<p id="t148" class="mis show_mis"><span class="n"><a href="#t148">148</a></span><span class="t"> <span class="key">if</span> <span class="nam">dbName</span> <span class="op">==</span> <span class="str">"External"</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t149" class="mis show_mis"><span class="n"><a href="#t149">149</a></span><span class="t"> <span class="nam">dbUnknown</span> <span class="op">=</span> <span class="key">True</span> </span><span class="r"></span></p>
<p id="t150" class="mis show_mis"><span class="n"><a href="#t150">150</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\tthe database for the externally referenced object is unknown"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t151" class="pln"><span class="n"><a href="#t151">151</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t152" class="pln"><span class="n"><a href="#t152">152</a></span><span class="t"> <span class="com"># note: if the database type is oracle, we can't trust the name of the database</span> </span><span class="r"></span></p>
<p id="t153" class="pln"><span class="n"><a href="#t153">153</a></span><span class="t"> <span class="com"># since it will contain the dblink name, not the externally referenced db</span> </span><span class="r"></span></p>
<p id="t154" class="pln"><span class="n"><a href="#t154">154</a></span><span class="t"> <span class="com"># so we will treat it as unknown and just try to find the schema/table ref'd</span> </span><span class="r"></span></p>
<p id="t155" class="mis show_mis"><span class="n"><a href="#t155">155</a></span><span class="t"> <span class="key">if</span> <span class="nam">resType</span> <span class="op">==</span> <span class="str">"Oracle"</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t156" class="mis show_mis"><span class="n"><a href="#t156">156</a></span><span class="t"> <span class="nam">dbUnknown</span> <span class="op">=</span> <span class="key">True</span> </span><span class="r"></span></p>
<p id="t157" class="pln"><span class="n"><a href="#t157">157</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t158" class="mis show_mis"><span class="n"><a href="#t158">158</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"\ttype={classType} name={dbName} id={dbId} unknownDB:{dbUnknown}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t159" class="pln"><span class="n"><a href="#t159">159</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t160" class="pln"><span class="n"><a href="#t160">160</a></span><span class="t"> <span class="com"># get the lineage for the database object</span> </span><span class="r"></span></p>
<p id="t161" class="mis show_mis"><span class="n"><a href="#t161">161</a></span><span class="t"> <span class="nam">lineageURL</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">baseUrl</span> <span class="op">+</span> <span class="str">"/access/2/catalog/data/relationships"</span> </span><span class="r"></span></p>
<p id="t162" class="mis show_mis"><span class="n"><a href="#t162">162</a></span><span class="t"> <span class="nam">lineageParms</span> <span class="op">=</span> <span class="op">{</span> </span><span class="r"></span></p>
<p id="t163" class="pln"><span class="n"><a href="#t163">163</a></span><span class="t"> <span class="str">"seed"</span><span class="op">:</span> <span class="nam">dbId</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t164" class="pln"><span class="n"><a href="#t164">164</a></span><span class="t"> <span class="str">"association"</span><span class="op">:</span> <span class="str">"core.ParentChild"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t165" class="pln"><span class="n"><a href="#t165">165</a></span><span class="t"> <span class="str">"depth"</span><span class="op">:</span> <span class="str">"2"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t166" class="pln"><span class="n"><a href="#t166">166</a></span><span class="t"> <span class="str">"direction"</span><span class="op">:</span> <span class="str">"OUT"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t167" class="pln"><span class="n"><a href="#t167">167</a></span><span class="t"> <span class="str">"includeAttribute"</span><span class="op">:</span> <span class="op">{</span><span class="str">"core.name"</span><span class="op">,</span> <span class="str">"core.classType"</span><span class="op">}</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t168" class="pln"><span class="n"><a href="#t168">168</a></span><span class="t"> <span class="str">"includeTerms"</span><span class="op">:</span> <span class="str">"false"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t169" class="pln"><span class="n"><a href="#t169">169</a></span><span class="t"> <span class="str">"removeDuplicateAggregateLinks"</span><span class="op">:</span> <span class="str">"false"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t170" class="pln"><span class="n"><a href="#t170">170</a></span><span class="t"> <span class="op">}</span> </span><span class="r"></span></p>
<p id="t171" class="mis show_mis"><span class="n"><a href="#t171">171</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"\tLineage query for: {dbName} params={lineageParms}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t172" class="mis show_mis"><span class="n"><a href="#t172">172</a></span><span class="t"> <span class="nam">lineageResp</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">session</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t173" class="pln"><span class="n"><a href="#t173">173</a></span><span class="t"> <span class="nam">lineageURL</span><span class="op">,</span> <span class="nam">params</span><span class="op">=</span><span class="nam">lineageParms</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t174" class="pln"><span class="n"><a href="#t174">174</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t175" class="pln"><span class="n"><a href="#t175">175</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t176" class="mis show_mis"><span class="n"><a href="#t176">176</a></span><span class="t"> <span class="nam">lineageStatus</span> <span class="op">=</span> <span class="nam">lineageResp</span><span class="op">.</span><span class="nam">status_code</span> </span><span class="r"></span></p>
<p id="t177" class="mis show_mis"><span class="n"><a href="#t177">177</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"\tlineage rc={lineageStatus}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t178" class="mis show_mis"><span class="n"><a href="#t178">178</a></span><span class="t"> <span class="nam">lineageJson</span> <span class="op">=</span> <span class="nam">lineageResp</span><span class="op">.</span><span class="nam">text</span> </span><span class="r"></span></p>
<p id="t179" class="pln"><span class="n"><a href="#t179">179</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t180" class="pln"><span class="n"><a href="#t180">180</a></span><span class="t"> <span class="com"># bug in the relationships api call - the items collection sould be</span> </span><span class="r"></span></p>
<p id="t181" class="pln"><span class="n"><a href="#t181">181</a></span><span class="t"> <span class="com"># "items" (with quotes)</span> </span><span class="r"></span></p>
<p id="t182" class="mis show_mis"><span class="n"><a href="#t182">182</a></span><span class="t"> <span class="key">if</span> <span class="nam">lineageJson</span><span class="op">.</span><span class="nam">startswith</span><span class="op">(</span><span class="str">"{items"</span><span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t183" class="pln"><span class="n"><a href="#t183">183</a></span><span class="t"> <span class="com"># bug in 10.2.1 and before</span> </span><span class="r"></span></p>
<p id="t184" class="pln"><span class="n"><a href="#t184">184</a></span><span class="t"> <span class="com"># replace items with "items" - for just the first occurrence</span> </span><span class="r"></span></p>
<p id="t185" class="pln"><span class="n"><a href="#t185">185</a></span><span class="t"> <span class="com"># (note: this is fixed after 10.2.1</span> </span><span class="r"></span></p>
<p id="t186" class="mis show_mis"><span class="n"><a href="#t186">186</a></span><span class="t"> <span class="nam">lineageJson</span> <span class="op">=</span> <span class="nam">json</span><span class="op">.</span><span class="nam">loads</span><span class="op">(</span><span class="nam">lineageJson</span><span class="op">.</span><span class="nam">replace</span><span class="op">(</span><span class="str">"items"</span><span class="op">,</span> <span class="str">'"items"'</span><span class="op">,</span> <span class="num">1</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t187" class="pln"><span class="n"><a href="#t187">187</a></span><span class="t"> <span class="key">else</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t188" class="pln"><span class="n"><a href="#t188">188</a></span><span class="t"> <span class="com"># 10.2.1u1 + json is escaped properly</span> </span><span class="r"></span></p>
<p id="t189" class="mis show_mis"><span class="n"><a href="#t189">189</a></span><span class="t"> <span class="nam">lineageJson</span> <span class="op">=</span> <span class="nam">json</span><span class="op">.</span><span class="nam">loads</span><span class="op">(</span><span class="nam">lineageJson</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t190" class="pln"><span class="n"><a href="#t190">190</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t191" class="pln"><span class="n"><a href="#t191">191</a></span><span class="t"> <span class="com"># for each item in the lineage resultset</span> </span><span class="r"></span></p>
<p id="t192" class="mis show_mis"><span class="n"><a href="#t192">192</a></span><span class="t"> <span class="key">for</span> <span class="nam">lineageItem</span> <span class="key">in</span> <span class="nam">lineageJson</span><span class="op">[</span><span class="str">"items"</span><span class="op">]</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t193" class="mis show_mis"><span class="n"><a href="#t193">193</a></span><span class="t"> <span class="nam">inId</span> <span class="op">=</span> <span class="nam">lineageItem</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"inId"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t194" class="mis show_mis"><span class="n"><a href="#t194">194</a></span><span class="t"> <span class="nam">assocId</span> <span class="op">=</span> <span class="nam">lineageItem</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"associationId"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t195" class="mis show_mis"><span class="n"><a href="#t195">195</a></span><span class="t"> <span class="nam">schemaName</span> <span class="op">=</span> <span class="str">""</span> </span><span class="r"></span></p>
<p id="t196" class="pln"><span class="n"><a href="#t196">196</a></span><span class="t"> <span class="com"># print("\t" + inId + " assoc=" + assocId)</span> </span><span class="r"></span></p>
<p id="t197" class="mis show_mis"><span class="n"><a href="#t197">197</a></span><span class="t"> <span class="key">if</span> <span class="nam">assocId</span> <span class="op">==</span> <span class="str">"com.infa.ldm.relational.ExternalSchemaTable"</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t198" class="pln"><span class="n"><a href="#t198">198</a></span><span class="t"> <span class="com"># find the table...</span> </span><span class="r"></span></p>
<p id="t199" class="mis show_mis"><span class="n"><a href="#t199">199</a></span><span class="t"> <span class="nam">schemaName</span> <span class="op">=</span> <span class="nam">inId</span><span class="op">.</span><span class="nam">split</span><span class="op">(</span><span class="str">"/"</span><span class="op">)</span><span class="op">[</span><span class="op">-</span><span class="num">2</span><span class="op">]</span> </span><span class="r"></span></p>
<p id="t200" class="mis show_mis"><span class="n"><a href="#t200">200</a></span><span class="t"> <span class="nam">inEmbedded</span> <span class="op">=</span> <span class="nam">lineageItem</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="str">"inEmbedded"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t201" class="mis show_mis"><span class="n"><a href="#t201">201</a></span><span class="t"> <span class="nam">tableName</span> <span class="op">=</span> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">get_fact_value</span><span class="op">(</span><span class="nam">inEmbedded</span><span class="op">,</span> <span class="str">"core.name"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t202" class="mis show_mis"><span class="n"><a href="#t202">202</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t203" class="pln"><span class="n"><a href="#t203">203</a></span><span class="t"> <span class="str">f"\tprocessing table={tableName} schema={schemaName}"</span> </span><span class="r"></span></p>
<p id="t204" class="pln"><span class="n"><a href="#t204">204</a></span><span class="t"> <span class="str">f" db={dbName} id={inId}"</span> </span><span class="r"></span></p>
<p id="t205" class="pln"><span class="n"><a href="#t205">205</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t206" class="pln"><span class="n"><a href="#t206">206</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t207" class="pln"><span class="n"><a href="#t207">207</a></span><span class="t"> <span class="com"># format the query to find the actual table</span> </span><span class="r"></span></p>
<p id="t208" class="pln"><span class="n"><a href="#t208">208</a></span><span class="t"> <span class="com"># "core.classType:(com.infa.ldm.relational.Table or com.infa.ldm.relational.View) and core.name_lc_exact:"</span> </span><span class="r"></span></p>
<p id="t209" class="mis show_mis"><span class="n"><a href="#t209">209</a></span><span class="t"> <span class="nam">q</span> <span class="op">=</span> <span class="op">(</span> </span><span class="r"></span></p>
<p id="t210" class="pln"><span class="n"><a href="#t210">210</a></span><span class="t"> <span class="str">"core.classType:(com.infa.ldm.relational.Table or com.infa.ldm.relational.View) and core.name:\""</span> </span><span class="r"></span></p>
<p id="t211" class="pln"><span class="n"><a href="#t211">211</a></span><span class="t"> <span class="op">+</span> <span class="nam">tableName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t212" class="pln"><span class="n"><a href="#t212">212</a></span><span class="t"> <span class="op">+</span> <span class="str">"\""</span> </span><span class="r"></span></p>
<p id="t213" class="pln"><span class="n"><a href="#t213">213</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t214" class="mis show_mis"><span class="n"><a href="#t214">214</a></span><span class="t"> <span class="key">if</span> <span class="nam">dbUnknown</span> <span class="key">and</span> <span class="nam">schemaName</span> <span class="op">==</span> <span class="str">""</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t215" class="mis show_mis"><span class="n"><a href="#t215">215</a></span><span class="t"> <span class="nam">q</span> <span class="op">=</span> <span class="nam">q</span> <span class="op">+</span> <span class="str">" and core.resource_name:"</span> <span class="op">+</span> <span class="nam">resName</span> </span><span class="r"></span></p>
<p id="t216" class="pln"><span class="n"><a href="#t216">216</a></span><span class="t"> <span class="com"># if dbUnknown==False:</span> </span><span class="r"></span></p>
<p id="t217" class="pln"><span class="n"><a href="#t217">217</a></span><span class="t"> <span class="com"># q=q+ ' and ' + dbName</span> </span><span class="r"></span></p>
<p id="t218" class="mis show_mis"><span class="n"><a href="#t218">218</a></span><span class="t"> <span class="nam">q</span> <span class="op">=</span> <span class="nam">q</span> <span class="op">+</span> <span class="str">' and core.resourceType:"'</span> <span class="op">+</span> <span class="nam">resType</span> <span class="op">+</span> <span class="str">'"'</span> </span><span class="r"></span></p>
<p id="t219" class="mis show_mis"><span class="n"><a href="#t219">219</a></span><span class="t"> <span class="nam">tableSearchParms</span> <span class="op">=</span> <span class="op">{</span><span class="str">"q"</span><span class="op">:</span> <span class="nam">q</span><span class="op">,</span> <span class="str">"offset"</span><span class="op">:</span> <span class="num">0</span><span class="op">,</span> <span class="str">"pageSize"</span><span class="op">:</span> <span class="num">100</span><span class="op">}</span> </span><span class="r"></span></p>
<p id="t220" class="mis show_mis"><span class="n"><a href="#t220">220</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\tquery="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">tableSearchParms</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t221" class="pln"><span class="n"><a href="#t221">221</a></span><span class="t"> <span class="com"># find the table - with name tableName</span> </span><span class="r"></span></p>
<p id="t222" class="mis show_mis"><span class="n"><a href="#t222">222</a></span><span class="t"> <span class="nam">tResp</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">session</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t223" class="pln"><span class="n"><a href="#t223">223</a></span><span class="t"> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">baseUrl</span> <span class="op">+</span> <span class="str">"/access/2/catalog/data/objects"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t224" class="pln"><span class="n"><a href="#t224">224</a></span><span class="t"> <span class="nam">params</span><span class="op">=</span><span class="nam">tableSearchParms</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t225" class="pln"><span class="n"><a href="#t225">225</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t226" class="mis show_mis"><span class="n"><a href="#t226">226</a></span><span class="t"> <span class="nam">tStatus</span> <span class="op">=</span> <span class="nam">tResp</span><span class="op">.</span><span class="nam">status_code</span> </span><span class="r"></span></p>
<p id="t227" class="mis show_mis"><span class="n"><a href="#t227">227</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\tquery rc="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">tStatus</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t228" class="pln"><span class="n"><a href="#t228">228</a></span><span class="t"> <span class="com"># print("\t\t\t\t" + str(tResp.json()))</span> </span><span class="r"></span></p>
<p id="t229" class="mis show_mis"><span class="n"><a href="#t229">229</a></span><span class="t"> <span class="nam">foundTabId</span> <span class="op">=</span> <span class="str">""</span> </span><span class="r"></span></p>
<p id="t230" class="mis show_mis"><span class="n"><a href="#t230">230</a></span><span class="t"> <span class="nam">tableMatchCount</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t231" class="pln"><span class="n"><a href="#t231">231</a></span><span class="t"> <span class="com"># possible matching tables</span> </span><span class="r"></span></p>
<p id="t232" class="mis show_mis"><span class="n"><a href="#t232">232</a></span><span class="t"> <span class="key">for</span> <span class="nam">tableItem</span> <span class="key">in</span> <span class="nam">tResp</span><span class="op">.</span><span class="nam">json</span><span class="op">(</span><span class="op">)</span><span class="op">[</span><span class="str">"items"</span><span class="op">]</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t233" class="mis show_mis"><span class="n"><a href="#t233">233</a></span><span class="t"> <span class="nam">fromTableId</span> <span class="op">=</span> <span class="nam">tableItem</span><span class="op">[</span><span class="str">"id"</span><span class="op">]</span> </span><span class="r"></span></p>
<p id="t234" class="mis show_mis"><span class="n"><a href="#t234">234</a></span><span class="t"> <span class="nam">fromSchemaName</span> <span class="op">=</span> <span class="nam">fromTableId</span><span class="op">.</span><span class="nam">split</span><span class="op">(</span><span class="str">"/"</span><span class="op">)</span><span class="op">[</span><span class="op">-</span><span class="num">2</span><span class="op">]</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t235" class="pln"><span class="n"><a href="#t235">235</a></span><span class="t"> <span class="com"># foundTabId=''</span> </span><span class="r"></span></p>
<p id="t236" class="pln"><span class="n"><a href="#t236">236</a></span><span class="t"> <span class="com"># foundTabId = tableItem["id"]</span> </span><span class="r"></span></p>
<p id="t237" class="pln"><span class="n"><a href="#t237">237</a></span><span class="t"> <span class="com"># print("rs-" + fromTableId + " compared with " + inId);</span> </span><span class="r"></span></p>
<p id="t238" class="mis show_mis"><span class="n"><a href="#t238">238</a></span><span class="t"> <span class="key">if</span> <span class="nam">fromTableId</span> <span class="op">!=</span> <span class="nam">inId</span> <span class="key">and</span> <span class="nam">fromTableId</span><span class="op">.</span><span class="nam">count</span><span class="op">(</span><span class="str">"/"</span><span class="op">)</span> <span class="op">==</span> <span class="num">4</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t239" class="pln"><span class="n"><a href="#t239">239</a></span><span class="t"> <span class="com"># filter out comparing against itself +</span> </span><span class="r"></span></p>
<p id="t240" class="pln"><span class="n"><a href="#t240">240</a></span><span class="t"> <span class="com"># counting the / chars - 4 = normal db, 5=external db (disregard)</span> </span><span class="r"></span></p>
<p id="t241" class="mis show_mis"><span class="n"><a href="#t241">241</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\tchecking "</span> <span class="op">+</span> <span class="nam">fromTableId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t242" class="pln"><span class="n"><a href="#t242">242</a></span><span class="t"> <span class="com"># print("could be this one...." + foundTabId + " << " + inId)</span> </span><span class="r"></span></p>
<p id="t243" class="mis show_mis"><span class="n"><a href="#t243">243</a></span><span class="t"> <span class="nam">theName</span> <span class="op">=</span> <span class="nam">fromTableId</span><span class="op">.</span><span class="nam">split</span><span class="op">(</span><span class="str">"/"</span><span class="op">)</span><span class="op">[</span><span class="op">-</span><span class="num">1</span><span class="op">]</span> </span><span class="r"></span></p>
<p id="t244" class="pln"><span class="n"><a href="#t244">244</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t245" class="pln"><span class="n"><a href="#t245">245</a></span><span class="t"> <span class="com"># the schema could be empty - in that case it should match dbo</span> </span><span class="r"></span></p>
<p id="t246" class="pln"><span class="n"><a href="#t246">246</a></span><span class="t"> <span class="com"># unless we have a way of knowing what the default schema is</span> </span><span class="r"></span></p>
<p id="t247" class="mis show_mis"><span class="n"><a href="#t247">247</a></span><span class="t"> <span class="key">if</span> <span class="nam">theName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> <span class="op">==</span> <span class="nam">tableName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> <span class="key">and</span> <span class="op">(</span> </span><span class="r"></span></p>
<p id="t248" class="pln"><span class="n"><a href="#t248">248</a></span><span class="t"> <span class="nam">schemaName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> <span class="op">==</span> <span class="nam">fromSchemaName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t249" class="pln"><span class="n"><a href="#t249">249</a></span><span class="t"> <span class="key">or</span> <span class="op">(</span><span class="nam">schemaName</span> <span class="op">==</span> <span class="str">""</span> <span class="key">and</span> <span class="nam">fromSchemaName</span> <span class="op">==</span> <span class="str">"dbo"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t250" class="pln"><span class="n"><a href="#t250">250</a></span><span class="t"> <span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t251" class="mis show_mis"><span class="n"><a href="#t251">251</a></span><span class="t"> <span class="nam">tableMatchCount</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t252" class="mis show_mis"><span class="n"><a href="#t252">252</a></span><span class="t"> <span class="nam">foundTabId</span> <span class="op">=</span> <span class="nam">fromTableId</span> </span><span class="r"></span></p>
<p id="t253" class="mis show_mis"><span class="n"><a href="#t253">253</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t254" class="pln"><span class="n"><a href="#t254">254</a></span><span class="t"> <span class="str">f"\t\ttable name matches...{theName}=={tableName} {inId} {foundTabId} "</span> </span><span class="r"></span></p>
<p id="t255" class="pln"><span class="n"><a href="#t255">255</a></span><span class="t"> <span class="str">f" count/{fromTableId.count('/')}"</span> </span><span class="r"></span></p>
<p id="t256" class="pln"><span class="n"><a href="#t256">256</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t257" class="pln"><span class="n"><a href="#t257">257</a></span><span class="t"> <span class="key">else</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t258" class="mis show_mis"><span class="n"><a href="#t258">258</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t259" class="pln"><span class="n"><a href="#t259">259</a></span><span class="t"> <span class="str">"\t\tno match...schema match 1:"</span> </span><span class="r"></span></p>
<p id="t260" class="pln"><span class="n"><a href="#t260">260</a></span><span class="t"> <span class="op">+</span> <span class="nam">theName</span> </span><span class="r"></span></p>
<p id="t261" class="pln"><span class="n"><a href="#t261">261</a></span><span class="t"> <span class="op">+</span> <span class="str">" 2:"</span> </span><span class="r"></span></p>
<p id="t262" class="pln"><span class="n"><a href="#t262">262</a></span><span class="t"> <span class="op">+</span> <span class="nam">tableName</span> </span><span class="r"></span></p>
<p id="t263" class="pln"><span class="n"><a href="#t263">263</a></span><span class="t"> <span class="op">+</span> <span class="str">" 3:"</span> </span><span class="r"></span></p>
<p id="t264" class="pln"><span class="n"><a href="#t264">264</a></span><span class="t"> <span class="op">+</span> <span class="nam">schemaName</span> </span><span class="r"></span></p>
<p id="t265" class="pln"><span class="n"><a href="#t265">265</a></span><span class="t"> <span class="op">+</span> <span class="str">" 4:"</span> </span><span class="r"></span></p>
<p id="t266" class="pln"><span class="n"><a href="#t266">266</a></span><span class="t"> <span class="op">+</span> <span class="nam">fromSchemaName</span> </span><span class="r"></span></p>
<p id="t267" class="pln"><span class="n"><a href="#t267">267</a></span><span class="t"> <span class="op">+</span> <span class="str">":"</span> </span><span class="r"></span></p>
<p id="t268" class="pln"><span class="n"><a href="#t268">268</a></span><span class="t"> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">schemaName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span> <span class="op">==</span> <span class="nam">fromSchemaName</span><span class="op">.</span><span class="nam">lower</span><span class="op">(</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t269" class="pln"><span class="n"><a href="#t269">269</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t270" class="pln"><span class="n"><a href="#t270">270</a></span><span class="t"> <span class="com"># else:</span> </span><span class="r"></span></p>
<p id="t271" class="pln"><span class="n"><a href="#t271">271</a></span><span class="t"> <span class="com"># print("skipping this one");</span> </span><span class="r"></span></p>
<p id="t272" class="pln"><span class="n"><a href="#t272">272</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t273" class="mis show_mis"><span class="n"><a href="#t273">273</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\ttotal matching tables="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">tableMatchCount</span><span class="op">)</span> <span class="op">+</span> <span class="str">" inId:"</span> <span class="op">+</span> <span class="nam">inId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t274" class="mis show_mis"><span class="n"><a href="#t274">274</a></span><span class="t"> <span class="key">if</span> <span class="nam">tableMatchCount</span> <span class="op">==</span> <span class="num">1</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t275" class="pln"><span class="n"><a href="#t275">275</a></span><span class="t"> <span class="com"># ok - we have a single table to match</span> </span><span class="r"></span></p>
<p id="t276" class="pln"><span class="n"><a href="#t276">276</a></span><span class="t"> <span class="com"># link at the table level - then get the columns and link them too</span> </span><span class="r"></span></p>
<p id="t277" class="pln"><span class="n"><a href="#t277">277</a></span><span class="t"> <span class="com"># print("linking from actual tab - to external ref tab")</span> </span><span class="r"></span></p>
<p id="t278" class="mis show_mis"><span class="n"><a href="#t278">278</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"\t\tlinking tables "</span> <span class="op">+</span> <span class="nam">foundTabId</span> <span class="op">+</span> <span class="str">" -->> "</span> <span class="op">+</span> <span class="nam">inId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t279" class="pln"><span class="n"><a href="#t279">279</a></span><span class="t"> <span class="com"># get the columns for the ext Table</span> </span><span class="r"></span></p>
<p id="t280" class="pln"><span class="n"><a href="#t280">280</a></span><span class="t"> <span class="com"># (will be a reduced set - only the linked columns)</span> </span><span class="r"></span></p>
<p id="t281" class="mis show_mis"><span class="n"><a href="#t281">281</a></span><span class="t"> <span class="nam">extCols</span> <span class="op">=</span> <span class="nam">getColumnsForTable</span><span class="op">(</span><span class="nam">inId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t282" class="mis show_mis"><span class="n"><a href="#t282">282</a></span><span class="t"> <span class="nam">tabCols</span> <span class="op">=</span> <span class="nam">getColumnsForTable</span><span class="op">(</span><span class="nam">foundTabId</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t283" class="pln"><span class="n"><a href="#t283">283</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t284" class="pln"><span class="n"><a href="#t284">284</a></span><span class="t"> <span class="com"># link the table level</span> </span><span class="r"></span></p>
<p id="t285" class="mis show_mis"><span class="n"><a href="#t285">285</a></span><span class="t"> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">exportLineageLink</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t286" class="pln"><span class="n"><a href="#t286">286</a></span><span class="t"> <span class="nam">foundTabId</span><span class="op">,</span> <span class="nam">inId</span><span class="op">,</span> <span class="str">"core.DataSetDataFlow"</span><span class="op">,</span> <span class="nam">colWriter</span> </span><span class="r"></span></p>
<p id="t287" class="pln"><span class="n"><a href="#t287">287</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t288" class="mis show_mis"><span class="n"><a href="#t288">288</a></span><span class="t"> <span class="nam">tabLinks</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t289" class="mis show_mis"><span class="n"><a href="#t289">289</a></span><span class="t"> <span class="nam">tabColsLinked</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t290" class="pln"><span class="n"><a href="#t290">290</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t291" class="pln"><span class="n"><a href="#t291">291</a></span><span class="t"> <span class="com"># match the columns on the left/right side</span> </span><span class="r"></span></p>
<p id="t292" class="mis show_mis"><span class="n"><a href="#t292">292</a></span><span class="t"> <span class="key">for</span> <span class="nam">toCol</span><span class="op">,</span> <span class="nam">toId</span> <span class="key">in</span> <span class="nam">extCols</span><span class="op">.</span><span class="nam">items</span><span class="op">(</span><span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t293" class="pln"><span class="n"><a href="#t293">293</a></span><span class="t"> <span class="com"># check if the toCol (the name) exists in the tabCols dict</span> </span><span class="r"></span></p>
<p id="t294" class="pln"><span class="n"><a href="#t294">294</a></span><span class="t"> <span class="com"># print('\t\t\tchecking toCol:' + toCol + tabCols.get(toCol))</span> </span><span class="r"></span></p>
<p id="t295" class="mis show_mis"><span class="n"><a href="#t295">295</a></span><span class="t"> <span class="nam">fromId</span> <span class="op">=</span> <span class="nam">tabCols</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="nam">toCol</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t296" class="mis show_mis"><span class="n"><a href="#t296">296</a></span><span class="t"> <span class="key">if</span> <span class="nam">fromId</span> <span class="key">is</span> <span class="key">not</span> <span class="key">None</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t297" class="pln"><span class="n"><a href="#t297">297</a></span><span class="t"> <span class="com"># print('\t\t\tlinking columns...' + fromId + ' --->>>' + toId)</span> </span><span class="r"></span></p>
<p id="t298" class="mis show_mis"><span class="n"><a href="#t298">298</a></span><span class="t"> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">exportLineageLink</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t299" class="pln"><span class="n"><a href="#t299">299</a></span><span class="t"> <span class="nam">fromId</span><span class="op">,</span> <span class="nam">toId</span><span class="op">,</span> <span class="str">"core.DirectionalDataFlow"</span><span class="op">,</span> <span class="nam">colWriter</span> </span><span class="r"></span></p>
<p id="t300" class="pln"><span class="n"><a href="#t300">300</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t301" class="mis show_mis"><span class="n"><a href="#t301">301</a></span><span class="t"> <span class="nam">tabColsLinked</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t302" class="mis show_mis"><span class="n"><a href="#t302">302</a></span><span class="t"> <span class="nam">colLinks</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t303" class="pln"><span class="n"><a href="#t303">303</a></span><span class="t"> <span class="key">else</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t304" class="mis show_mis"><span class="n"><a href="#t304">304</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t305" class="pln"><span class="n"><a href="#t305">305</a></span><span class="t"> <span class="str">f"\t\t\tError: cannot find column {toCol} in table {inId}"</span> </span><span class="r"></span></p>
<p id="t306" class="pln"><span class="n"><a href="#t306">306</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t307" class="mis show_mis"><span class="n"><a href="#t307">307</a></span><span class="t"> <span class="nam">errors</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t308" class="mis show_mis"><span class="n"><a href="#t308">308</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t309" class="pln"><span class="n"><a href="#t309">309</a></span><span class="t"> <span class="str">f"\t\t\text cols:{len(extCols)} tablCols:{len(tabCols)}"</span> </span><span class="r"></span></p>
<p id="t310" class="pln"><span class="n"><a href="#t310">310</a></span><span class="t"> <span class="str">f" linked={tabColsLinked}"</span> </span><span class="r"></span></p>
<p id="t311" class="pln"><span class="n"><a href="#t311">311</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t312" class="pln"><span class="n"><a href="#t312">312</a></span><span class="t"> <span class="com"># print("\t\t\tcolumns linked=" + str(tabColsLinked))</span> </span><span class="r"></span></p>
<p id="t313" class="pln"><span class="n"><a href="#t313">313</a></span><span class="t"> <span class="key">else</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t314" class="mis show_mis"><span class="n"><a href="#t314">314</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t315" class="pln"><span class="n"><a href="#t315">315</a></span><span class="t"> <span class="str">f"\t\tmutlple possible matches found ({tableMatchCount}"</span> </span><span class="r"></span></p>
<p id="t316" class="pln"><span class="n"><a href="#t316">316</a></span><span class="t"> <span class="str">f") no links will be created"</span> </span><span class="r"></span></p>
<p id="t317" class="pln"><span class="n"><a href="#t317">317</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t318" class="pln"><span class="n"><a href="#t318">318</a></span><span class="t"> <span class="com"># flush the console buffer - for tailing the stdout log</span> </span><span class="r"></span></p>
<p id="t319" class="mis show_mis"><span class="n"><a href="#t319">319</a></span><span class="t"> <span class="nam">sys</span><span class="op">.</span><span class="nam">stdout</span><span class="op">.</span><span class="nam">flush</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t320" class="pln"><span class="n"><a href="#t320">320</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t321" class="mis show_mis"><span class="n"><a href="#t321">321</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t322" class="pln"><span class="n"><a href="#t322">322</a></span><span class="t"> <span class="str">f"external database: {dbName} processed: tab/col links created: "</span> </span><span class="r"></span></p>
<p id="t323" class="pln"><span class="n"><a href="#t323">323</a></span><span class="t"> <span class="str">f"{tabLinks}/{colLinks} errors:{errors}"</span> </span><span class="r"></span></p>
<p id="t324" class="pln"><span class="n"><a href="#t324">324</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t325" class="mis show_mis"><span class="n"><a href="#t325">325</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">""</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t326" class="mis show_mis"><span class="n"><a href="#t326">326</a></span><span class="t"> <span class="key">return</span> <span class="nam">tabLinks</span><span class="op">,</span> <span class="nam">colLinks</span><span class="op">,</span> <span class="nam">errors</span> </span><span class="r"></span></p>
<p id="t327" class="pln"><span class="n"><a href="#t327">327</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t328" class="pln"><span class="n"><a href="#t328">328</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t329" class="mis show_mis"><span class="n"><a href="#t329">329</a></span><span class="t"><span class="key">def</span> <span class="nam">main</span><span class="op">(</span><span class="op">)</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t330" class="pln"><span class="n"><a href="#t330">330</a></span><span class="t"> <span class="str">"""</span> </span><span class="r"></span></p>
<p id="t331" class="pln"><span class="n"><a href="#t331">331</a></span><span class="t"><span class="str"> main starts here - run the query processing all items</span> </span><span class="r"></span></p>
<p id="t332" class="pln"><span class="n"><a href="#t332">332</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t333" class="pln"><span class="n"><a href="#t333">333</a></span><span class="t"><span class="str"> - find all com.infa.ldm.relational.ExternalDatabase objects</span> </span><span class="r"></span></p>
<p id="t334" class="pln"><span class="n"><a href="#t334">334</a></span><span class="t"><span class="str"> - for each - call processExternalDB</span> </span><span class="r"></span></p>
<p id="t335" class="pln"><span class="n"><a href="#t335">335</a></span><span class="t"><span class="str"> """</span> </span><span class="r"></span></p>
<p id="t336" class="mis show_mis"><span class="n"><a href="#t336">336</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"ExternalDBLinker started"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t337" class="mis show_mis"><span class="n"><a href="#t337">337</a></span><span class="t"> <span class="nam">start_time</span> <span class="op">=</span> <span class="nam">time</span><span class="op">.</span><span class="nam">time</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t338" class="pln"><span class="n"><a href="#t338">338</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t339" class="mis show_mis"><span class="n"><a href="#t339">339</a></span><span class="t"> <span class="nam">args</span> <span class="op">=</span> <span class="nam">args</span><span class="op">,</span> <span class="nam">unknown</span> <span class="op">=</span> <span class="nam">parser</span><span class="op">.</span><span class="nam">parse_known_args</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t340" class="pln"><span class="n"><a href="#t340">340</a></span><span class="t"> <span class="com"># setup edc session and catalog url - with auth in the session header,</span> </span><span class="r"></span></p>
<p id="t341" class="pln"><span class="n"><a href="#t341">341</a></span><span class="t"> <span class="com"># by using system vars or command-line args</span> </span><span class="r"></span></p>
<p id="t342" class="mis show_mis"><span class="n"><a href="#t342">342</a></span><span class="t"> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">init_edc_session</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t343" class="mis show_mis"><span class="n"><a href="#t343">343</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"command-line args parsed = {args} "</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t344" class="pln"><span class="n"><a href="#t344">344</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t345" class="mis show_mis"><span class="n"><a href="#t345">345</a></span><span class="t"> <span class="nam">tableLinksCreated</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t346" class="mis show_mis"><span class="n"><a href="#t346">346</a></span><span class="t"> <span class="nam">columnLinksCreated</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t347" class="mis show_mis"><span class="n"><a href="#t347">347</a></span><span class="t"> <span class="nam">errorsFound</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t348" class="pln"><span class="n"><a href="#t348">348</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t349" class="mis show_mis"><span class="n"><a href="#t349">349</a></span><span class="t"> <span class="nam">columnHeader</span> <span class="op">=</span> <span class="op">[</span> </span><span class="r"></span></p>
<p id="t350" class="pln"><span class="n"><a href="#t350">350</a></span><span class="t"> <span class="str">"Association"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t351" class="pln"><span class="n"><a href="#t351">351</a></span><span class="t"> <span class="str">"From Connection"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t352" class="pln"><span class="n"><a href="#t352">352</a></span><span class="t"> <span class="str">"To Connection"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t353" class="pln"><span class="n"><a href="#t353">353</a></span><span class="t"> <span class="str">"From Object"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t354" class="pln"><span class="n"><a href="#t354">354</a></span><span class="t"> <span class="str">"To Object"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t355" class="pln"><span class="n"><a href="#t355">355</a></span><span class="t"> <span class="op">]</span> </span><span class="r"></span></p>
<p id="t356" class="mis show_mis"><span class="n"><a href="#t356">356</a></span><span class="t"> <span class="nam">outputFile</span> <span class="op">=</span> <span class="nam">args</span><span class="op">.</span><span class="nam">outDir</span> <span class="op">+</span> <span class="str">"/"</span> <span class="op">+</span> <span class="nam">args</span><span class="op">.</span><span class="nam">csv_file_name</span> </span><span class="r"></span></p>
<p id="t357" class="mis show_mis"><span class="n"><a href="#t357">357</a></span><span class="t"> <span class="nam">fullpath</span> <span class="op">=</span> <span class="nam">os</span><span class="op">.</span><span class="nam">path</span><span class="op">.</span><span class="nam">abspath</span><span class="op">(</span><span class="nam">outputFile</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t358" class="mis show_mis"><span class="n"><a href="#t358">358</a></span><span class="t"> <span class="nam">fCSVFile</span> <span class="op">=</span> <span class="nam">open</span><span class="op">(</span><span class="nam">outputFile</span><span class="op">,</span> <span class="str">"w"</span><span class="op">,</span> <span class="nam">newline</span><span class="op">=</span><span class="str">""</span><span class="op">,</span> <span class="nam">encoding</span><span class="op">=</span><span class="str">"utf-8"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t359" class="pln"><span class="n"><a href="#t359">359</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t360" class="mis show_mis"><span class="n"><a href="#t360">360</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"custom lineage file initialized. "</span> <span class="op">+</span> <span class="nam">outputFile</span> <span class="op">+</span> <span class="str">" RELATIVE="</span> <span class="op">+</span><span class="nam">fullpath</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t361" class="mis show_mis"><span class="n"><a href="#t361">361</a></span><span class="t"> <span class="nam">colWriter</span> <span class="op">=</span> <span class="nam">csv</span><span class="op">.</span><span class="nam">writer</span><span class="op">(</span><span class="nam">fCSVFile</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t362" class="mis show_mis"><span class="n"><a href="#t362">362</a></span><span class="t"> <span class="nam">colWriter</span><span class="op">.</span><span class="nam">writerow</span><span class="op">(</span><span class="nam">columnHeader</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t363" class="pln"><span class="n"><a href="#t363">363</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t364" class="mis show_mis"><span class="n"><a href="#t364">364</a></span><span class="t"> <span class="nam">parameters</span> <span class="op">=</span> <span class="op">{</span> </span><span class="r"></span></p>
<p id="t365" class="pln"><span class="n"><a href="#t365">365</a></span><span class="t"> <span class="str">"q"</span><span class="op">:</span> <span class="str">"core.classType:com.infa.ldm.relational.ExternalDatabase"</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t366" class="pln"><span class="n"><a href="#t366">366</a></span><span class="t"> <span class="str">"offset"</span><span class="op">:</span> <span class="num">0</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t367" class="pln"><span class="n"><a href="#t367">367</a></span><span class="t"> <span class="str">"pageSize"</span><span class="op">:</span> <span class="num">1000</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t368" class="pln"><span class="n"><a href="#t368">368</a></span><span class="t"> <span class="op">}</span> </span><span class="r"></span></p>
<p id="t369" class="mis show_mis"><span class="n"><a href="#t369">369</a></span><span class="t"> <span class="nam">url</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">baseUrl</span> <span class="op">+</span> <span class="str">"/access/2/catalog/data/objects"</span> </span><span class="r"></span></p>
<p id="t370" class="pln"><span class="n"><a href="#t370">370</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t371" class="mis show_mis"><span class="n"><a href="#t371">371</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t372" class="pln"><span class="n"><a href="#t372">372</a></span><span class="t"> <span class="str">"executing query to find all external DB objects: "</span> </span><span class="r"></span></p>
<p id="t373" class="pln"><span class="n"><a href="#t373">373</a></span><span class="t"> <span class="str">f"{url} q={parameters.get('q')} {parameters}"</span> </span><span class="r"></span></p>
<p id="t374" class="pln"><span class="n"><a href="#t374">374</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t375" class="mis show_mis"><span class="n"><a href="#t375">375</a></span><span class="t"> <span class="nam">resp</span> <span class="op">=</span> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">session</span><span class="op">.</span><span class="nam">get</span><span class="op">(</span><span class="nam">url</span><span class="op">,</span> <span class="nam">params</span><span class="op">=</span><span class="nam">parameters</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t376" class="mis show_mis"><span class="n"><a href="#t376">376</a></span><span class="t"> <span class="nam">status</span> <span class="op">=</span> <span class="nam">resp</span><span class="op">.</span><span class="nam">status_code</span> </span><span class="r"></span></p>
<p id="t377" class="mis show_mis"><span class="n"><a href="#t377">377</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"extDB query rc="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">status</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t378" class="pln"><span class="n"><a href="#t378">378</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t379" class="mis show_mis"><span class="n"><a href="#t379">379</a></span><span class="t"> <span class="key">if</span> <span class="nam">status</span> <span class="op">!=</span> <span class="num">200</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t380" class="mis show_mis"><span class="n"><a href="#t380">380</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"error - expecting 200 rc, got {status} - message={resp.json()}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t381" class="mis show_mis"><span class="n"><a href="#t381">381</a></span><span class="t"> <span class="key">return</span> </span><span class="r"></span></p>
<p id="t382" class="pln"><span class="n"><a href="#t382">382</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t383" class="mis show_mis"><span class="n"><a href="#t383">383</a></span><span class="t"> <span class="nam">resultJson</span> <span class="op">=</span> <span class="nam">resp</span><span class="op">.</span><span class="nam">json</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t384" class="mis show_mis"><span class="n"><a href="#t384">384</a></span><span class="t"> <span class="nam">total</span> <span class="op">=</span> <span class="nam">resultJson</span><span class="op">[</span><span class="str">"metadata"</span><span class="op">]</span><span class="op">[</span><span class="str">"totalCount"</span><span class="op">]</span> </span><span class="r"></span></p>
<p id="t385" class="mis show_mis"><span class="n"><a href="#t385">385</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"external db objects found... {total}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t386" class="mis show_mis"><span class="n"><a href="#t386">386</a></span><span class="t"> <span class="nam">currentDB</span> <span class="op">=</span> <span class="num">0</span> </span><span class="r"></span></p>
<p id="t387" class="pln"><span class="n"><a href="#t387">387</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t388" class="pln"><span class="n"><a href="#t388">388</a></span><span class="t"> <span class="com"># for each externalDatabase object</span> </span><span class="r"></span></p>
<p id="t389" class="mis show_mis"><span class="n"><a href="#t389">389</a></span><span class="t"> <span class="key">for</span> <span class="nam">extDBItem</span> <span class="key">in</span> <span class="nam">resultJson</span><span class="op">[</span><span class="str">"items"</span><span class="op">]</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t390" class="mis show_mis"><span class="n"><a href="#t390">390</a></span><span class="t"> <span class="nam">itemId</span> <span class="op">=</span> <span class="nam">extDBItem</span><span class="op">[</span><span class="str">"id"</span><span class="op">]</span> </span><span class="r"></span></p>
<p id="t391" class="mis show_mis"><span class="n"><a href="#t391">391</a></span><span class="t"> <span class="nam">currentDB</span> <span class="op">+=</span> <span class="num">1</span> </span><span class="r"></span></p>
<p id="t392" class="mis show_mis"><span class="n"><a href="#t392">392</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">f"processing database: {itemId} {currentDB} of {total}"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t393" class="mis show_mis"><span class="n"><a href="#t393">393</a></span><span class="t"> <span class="nam">itemType</span> <span class="op">=</span> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">get_fact_value</span><span class="op">(</span><span class="nam">extDBItem</span><span class="op">,</span> <span class="str">"core.classType"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t394" class="mis show_mis"><span class="n"><a href="#t394">394</a></span><span class="t"> <span class="nam">itemName</span> <span class="op">=</span> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">get_fact_value</span><span class="op">(</span><span class="nam">extDBItem</span><span class="op">,</span> <span class="str">"core.name"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t395" class="mis show_mis"><span class="n"><a href="#t395">395</a></span><span class="t"> <span class="nam">resourceName</span> <span class="op">=</span> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">get_fact_value</span><span class="op">(</span><span class="nam">extDBItem</span><span class="op">,</span> <span class="str">"core.resource_name"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t396" class="mis show_mis"><span class="n"><a href="#t396">396</a></span><span class="t"> <span class="nam">resourceType</span> <span class="op">=</span> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">get_fact_value</span><span class="op">(</span><span class="nam">extDBItem</span><span class="op">,</span> <span class="str">"core.resourceType"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t397" class="pln"><span class="n"><a href="#t397">397</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t398" class="mis show_mis"><span class="n"><a href="#t398">398</a></span><span class="t"> <span class="nam">tabLinks</span><span class="op">,</span> <span class="nam">colLinks</span><span class="op">,</span> <span class="nam">errors</span> <span class="op">=</span> <span class="nam">processExternalDB</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t399" class="pln"><span class="n"><a href="#t399">399</a></span><span class="t"> <span class="nam">itemId</span><span class="op">,</span> <span class="nam">itemType</span><span class="op">,</span> <span class="nam">itemName</span><span class="op">,</span> <span class="nam">resourceType</span><span class="op">,</span> <span class="nam">resourceName</span><span class="op">,</span> <span class="nam">colWriter</span> </span><span class="r"></span></p>
<p id="t400" class="pln"><span class="n"><a href="#t400">400</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t401" class="mis show_mis"><span class="n"><a href="#t401">401</a></span><span class="t"> <span class="nam">tableLinksCreated</span> <span class="op">+=</span> <span class="nam">tabLinks</span> </span><span class="r"></span></p>
<p id="t402" class="mis show_mis"><span class="n"><a href="#t402">402</a></span><span class="t"> <span class="nam">columnLinksCreated</span> <span class="op">+=</span> <span class="nam">colLinks</span> </span><span class="r"></span></p>
<p id="t403" class="mis show_mis"><span class="n"><a href="#t403">403</a></span><span class="t"> <span class="nam">errorsFound</span> <span class="op">+=</span> <span class="nam">errors</span> </span><span class="r"></span></p>
<p id="t404" class="pln"><span class="n"><a href="#t404">404</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t405" class="mis show_mis"><span class="n"><a href="#t405">405</a></span><span class="t"> <span class="nam">sys</span><span class="op">.</span><span class="nam">stdout</span><span class="op">.</span><span class="nam">flush</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t406" class="pln"><span class="n"><a href="#t406">406</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t407" class="mis show_mis"><span class="n"><a href="#t407">407</a></span><span class="t"> <span class="nam">fCSVFile</span><span class="op">.</span><span class="nam">close</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t408" class="mis show_mis"><span class="n"><a href="#t408">408</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"finished!"</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t409" class="mis show_mis"><span class="n"><a href="#t409">409</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"table links: created="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">tableLinksCreated</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t410" class="mis show_mis"><span class="n"><a href="#t410">410</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"column links: created="</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">columnLinksCreated</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t411" class="mis show_mis"><span class="n"><a href="#t411">411</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"errors found: "</span> <span class="op">+</span> <span class="nam">str</span><span class="op">(</span><span class="nam">errorsFound</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t412" class="pln"><span class="n"><a href="#t412">412</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t413" class="pln"><span class="n"><a href="#t413">413</a></span><span class="t"> <span class="com"># end of main()</span> </span><span class="r"></span></p>
<p id="t414" class="mis show_mis"><span class="n"><a href="#t414">414</a></span><span class="t"> <span class="nam">print</span><span class="op">(</span><span class="str">"Finished - run time = %s seconds ---"</span> <span class="op">%</span> <span class="op">(</span><span class="nam">time</span><span class="op">.</span><span class="nam">time</span><span class="op">(</span><span class="op">)</span> <span class="op">-</span> <span class="nam">start_time</span><span class="op">)</span><span class="op">)</span> </span><span class="r"></span></p>
<p id="t415" class="pln"><span class="n"><a href="#t415">415</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t416" class="pln"><span class="n"><a href="#t416">416</a></span><span class="t"> <span class="com"># call the resource create/update/load function to get the data imported into EDC</span> </span><span class="r"></span></p>
<p id="t417" class="mis show_mis"><span class="n"><a href="#t417">417</a></span><span class="t"> <span class="key">if</span> <span class="nam">args</span><span class="op">.</span><span class="nam">edcimport</span> <span class="key">and</span> <span class="nam">tableLinksCreated</span> <span class="op">></span> <span class="num">0</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t418" class="mis show_mis"><span class="n"><a href="#t418">418</a></span><span class="t"> <span class="nam">edcutils</span><span class="op">.</span><span class="nam">createOrUpdateAndExecuteResourceUsingSession</span><span class="op">(</span> </span><span class="r"></span></p>
<p id="t419" class="pln"><span class="n"><a href="#t419">419</a></span><span class="t"> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">baseUrl</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t420" class="pln"><span class="n"><a href="#t420">420</a></span><span class="t"> <span class="nam">edcHelper</span><span class="op">.</span><span class="nam">session</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t421" class="pln"><span class="n"><a href="#t421">421</a></span><span class="t"> <span class="nam">args</span><span class="op">.</span><span class="nam">lineageResourceName</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t422" class="pln"><span class="n"><a href="#t422">422</a></span><span class="t"> <span class="nam">args</span><span class="op">.</span><span class="nam">lineageResourceTemplate</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t423" class="pln"><span class="n"><a href="#t423">423</a></span><span class="t"> <span class="nam">args</span><span class="op">.</span><span class="nam">csv_file_name</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t424" class="pln"><span class="n"><a href="#t424">424</a></span><span class="t"> <span class="nam">fullpath</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t425" class="pln"><span class="n"><a href="#t425">425</a></span><span class="t"> <span class="nam">waitToComplete</span><span class="op">,</span> </span><span class="r"></span></p>
<p id="t426" class="pln"><span class="n"><a href="#t426">426</a></span><span class="t"> <span class="str">"LineageScanner"</span> </span><span class="r"></span></p>
<p id="t427" class="pln"><span class="n"><a href="#t427">427</a></span><span class="t"> <span class="op">)</span> </span><span class="r"></span></p>
<p id="t428" class="pln"><span class="n"><a href="#t428">428</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t429" class="pln"><span class="n"><a href="#t429">429</a></span><span class="t"> </span><span class="r"></span></p>
<p id="t430" class="pln"><span class="n"><a href="#t430">430</a></span><span class="t"><span class="com"># call main - if not already called or used by another script</span> </span><span class="r"></span></p>
<p id="t431" class="mis show_mis"><span class="n"><a href="#t431">431</a></span><span class="t"><span class="key">if</span> <span class="nam">__name__</span> <span class="op">==</span> <span class="str">"__main__"</span><span class="op">:</span> </span><span class="r"></span></p>
<p id="t432" class="mis show_mis"><span class="n"><a href="#t432">432</a></span><span class="t"> <span class="nam">main</span><span class="op">(</span><span class="op">)</span> </span><span class="r"></span></p>
</div>
<div id="footer">
<div class="content">
<p>
<a class="nav" href="index.html">« index</a> <a class="nav" href="https://coverage.readthedocs.io">coverage.py v5.3</a>,
created at 2020-12-08 16:27 +0100
</p>
</div>
</div>
</body>
</html>