Skip to content

Commit 328bcc5

Browse files
committed
T1.25b fix 1 docs: the spec's order and year rules, the register's counts, the round
date-rules.md says an open order is text_time_order in every reading and the year written is held to 1 to 9999. DT-15 counts 18,815 inference decisions and says how class a is checked; DT-16 takes the open orders in inference and the time column's choice and the 372 day-first warnings; DT-17's prose is corrected; DT-6 notes its reversal of owner decision 30, which the tracker's Journey row names for the owner. as-built, E68, the tracker row, its counter and the fix-round history record the round. Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
1 parent 56dfe63 commit 328bcc5

5 files changed

Lines changed: 53 additions & 27 deletions

File tree

‎docs/design/rust-core/phase-0/date-rules.md‎

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,7 @@ Its fields are parsed by `jiff`'s `strtime` with the format it is written in, wh
110110
- the date and the time of day must name one: a month of 1 to 12, a day the month has, an hour of 0 to 23, a minute and a second of 0 to 59 (no leap second), and an offset's hours 0 to 23 and minutes 0 to 59;
111111
- a two-digit year is placed in the hundred years around the clock's year, from 50 years before it to 49 after, the client's rule ("Two-digit years" above; the run's `HostFacts.now` in UTC); a year of three or four digits is the year written;
112112
- a value with a zone reads as the instant its wall clock names at that offset (`Z`, `UTC` and `GMT` at zero), one without as its wall clock in UTC;
113-
- the instant must lie in the years 1 to 9999, in UTC;
113+
- the year written must be 1 to 9999, and so must the instant's year, in UTC, so that no offset carries a year outside the list into it (`0000-12-31T23:00-02:00` is outside);
114114
- a value empty once its white space is gone holds no time, as a missing one.
115115

116116
Every other value is outside the list: dates in words (`March 3 2020`), packed digits (`20240101`), dots (`15.01.2024`), fields written short (`2024-1-5`), two separators (`2024/01-05`), a value naming no date (`2024-02-30`, `32-12-24`), a second of 60, a fraction past nine digits, an offset of a day or more or with minutes past 59, another zone name (`EST`, `utc`), the year 0, and words such as `now`, `today` or `NaT`.
@@ -127,9 +127,12 @@ Where the rule decides that the day comes first, inference warns (`DATES_READ_DA
127127
Each refusal is `UNSUPPORTED_REQUEST` naming its rule.
128128

129129
- `materialize` reads a fetched text time column by the reading its provider declares (`text_times`, register DT-7: `value_by_value` is this list): a value outside the list is refused (`text_time`), as are an order left open (`text_time_order`) and offsets mixed (`text_time_zones`, DT-11); a blank value is dropped and warned of (`TIME_VALUES_UNREAD`), as the reference drops it.
130-
- The time column's choice reads a text time column's preview to UTC, offsets mixed taken; a value outside the list, or an order left open, is refused (`text_time`) only where the choice asks where the column starts.
130+
- The time column's choice reads a text time column's preview to UTC, offsets mixed taken; a value outside the list (`text_time`), or an order left open (`text_time_order`), is refused only where the choice asks where the column starts.
131131
- Inference's score (`contains_timestamp`, "Where text becomes a date" above): 0.3 for a name that says time and the share of the first 100 non-empty values read as times, a timestamp at 1.0 or more.
132-
The share is counted at fewest, the fewer of the two orders' counts where the values decide none, and at most, the more of them counting the values in the client's shapes the list does not read and, under a name that says time, every value the list does not read; where the two decide otherwise the text is refused (`text_time`), and offsets mixed among the values read are refused (`text_time_zones`).
132+
The share is counted at fewest, the fewer of the two orders' counts where the values decide none.
133+
Where the values read in either order decide the score otherwise, the order is left open and the text is refused (`text_time_order`).
134+
Otherwise the share is counted at most too, adding the values in the client's shapes the list reads in neither order and, under a name that says time, every value the list does not read; where that decides otherwise the text is refused (`text_time`).
135+
Offsets mixed among the values read are refused (`text_time_zones`).
133136
So under a name that does not say time, text outside the client's shapes is no time, as the client's own inference holds (`client/core/dates.py`: inference does not read loose text as dates, which would read part codes as dates): `AW00011205` is no timestamp.
134137

135138
## The PQL literal rules

‎docs/design/rust-core/phase-1/as-built.md‎

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -225,16 +225,17 @@ Text times by a fixed list of formats (design, the T1.25b row; the owner's answe
225225

226226
As built at T1.25b (E68), each claim with its mechanism:
227227

228-
- The spec is one place: `date-rules.md`'s "The core's text formats" (the formats, what a value reads as, the order of day and month, the three readings); `sdm_engine::time::text`'s module comment points to it. `Reader::column` (with `each` its reading not to UTC), `share` and `day_first_format` follow it: `text_shapes` finds a value's shape (the client's two patterns, unchanged), `format` gives the `strtime` format of the list it is written in or nothing (two separators, a second past 59, a fraction past nine digits, an offset of a day or more or minutes past 59, a zone named other than `UTC` or `GMT`), and `jiff::fmt::strtime::parse` reads the fields, `BrokenDownTime::set_year` placing a two-digit year (`Reader::century`) and `to_datetime` refusing a day the month lacks; `decided_order` is the client's `day_comes_first`; an instant outside the years 1 to 9999 in UTC is outside. `text.rs` is 624 lines, 884 before (438 outside its tests, 706 before).
229-
- Held to the spec: `tests/text_times.rs` compares all 2,345 columns of `text-times.json` four ways (the values, to UTC, the share, the day-first format), none skipped, and fails closed on a perturbed instant, a refusal said read and a read said refused, another refusal named, a share or format changed, evidence missing, empty or retyped, and a run comparing fewer columns (`the_check_fails_closed`); `the_vectors_hold_every_answer` requires each kind of answer to be held. `text_times_record.py` reads the spec apart from the Rust reader, with Python's `re` and `datetime` only, every year-first ISO value cross-checked with `datetime.fromisoformat` to the microsecond. Seeded faults each fail the vectors (a format dropped, an offset's sign or minutes misread, the day-first rule flipped, the pivot moved; counts in the T1.25b commit text), and the Rust reader agrees with the Python reading on all 13,456 columns T1.25 recorded.
230-
- Against the reference: `text-times-differential.json`, recorded from pandas 3.0.6 by `text_times_differential.py` over T1.25's 13,456 columns, compares the four outcomes the reference reaches (a SQL source's value-by-value reading, the time column's choice, inference's decision under a name that says time and one that does not, the day-first warning) and classes every difference: `a`, refused outside the list (DT-15, approved by the owner's T1.25b answer); `b.order_open`, `b.order_decided`, `b.format_guess`, `b.offsets`, `b.range`, `b.order_rule` and their mixtures, values in the list read, refused or warned of otherwise (DT-16, proposed for the owner); `c`, inference's text outside the client's shapes under a name that does not say time (DT-17, proposed). Counts: materialize 1,667 of 13,456 agree, 10,623 `a`, 1,166 `b`; the time column's choice 1,630 agree, 10,623 `a`, 1,203 `b`; inference 7,236 of 26,908 agree, 18,856 `a`, 357 `b`, 459 `c`; the warning 2,438 of 2,810 agree, 361 `b`, 11 `c`. `the_differential_reports_the_core_as_it_reads` holds every example's core answer to the core, each reference answer to differ from it, `a`'s to be a named refusal, and the counts to add up. The old file's 2 MB of pandas readings are gone; `text_times_differential.py` is a protected path beside the recorder.
231-
- Inference: `infer::TextTimes` has `share` (fewest, most, and how many values lie outside the client's shapes) in place of `never_read`; `text_holds_dates` decides by the score at fewest and at most, the most counting every value outside the list under a name that says time (`names::says_time`), and refuses `text_time` where the two differ; `sdm-schema`'s `tests/time_choice.rs` holds the score's edges, the refusal where values could make it, codes under each kind of name, and offsets mixed with a stand-in reader. The time column's choice reads to UTC and refuses only where it asks where a column starts, as before. `DATES_READ_DAY_FIRST` names the first day-first value's format where the client's rule decides the day comes first (`Reader::day_first_format`, `assemble::day_first`).
228+
- The spec is one place: `date-rules.md`'s "The core's text formats" (the formats, what a value reads as, the order of day and month, the three readings); `sdm_engine::time::text`'s module comment points to it. `Reader::column` (with `each` its reading not to UTC), `share` and `day_first_format` follow it: `text_shapes` finds a value's shape (the client's two patterns, unchanged), `format` gives the `strtime` format of the list it is written in or nothing (two separators, a second past 59, a fraction past nine digits, an offset of a day or more or minutes past 59, a zone named other than `UTC` or `GMT`), and `jiff::fmt::strtime::parse` reads the fields, `BrokenDownTime::set_year` placing a two-digit year (`Reader::century`) and `to_datetime` refusing a day the month lacks; `decided_order` is the client's `day_comes_first`; a year written outside 1 to 9999, or an instant outside those years in UTC, is outside (the year written checked from fix round 1). `text.rs` is 624 lines, 884 before (438 outside its tests, 706 before).
229+
- Held to the spec: `tests/text_times.rs` compares all 2,360 columns of `text-times.json` four ways (the values, to UTC, the share, the day-first format), none skipped, and fails closed on a perturbed instant, a refusal said read and a read said refused, another refusal named, a share or format changed, evidence missing, empty or retyped, and a run comparing fewer columns (`the_check_fails_closed`); `the_vectors_hold_every_answer` requires each kind of answer to be held. `text_times_record.py` reads the spec apart from the Rust reader, with Python's `re` and `datetime` only, every year-first ISO value cross-checked with `datetime.fromisoformat` to the microsecond. Seeded faults each fail the vectors (a format dropped, an offset's sign or minutes misread, the day-first rule flipped, the pivot moved; counts in the T1.25b commit text), and the Rust reader agrees with the Python reading on all 13,456 columns T1.25 recorded.
230+
- Against the reference: `text-times-differential.json`, recorded from pandas 3.0.6 by `text_times_differential.py` over T1.25's 13,456 columns, compares the four outcomes the reference reaches (a SQL source's value-by-value reading, the time column's choice, inference's decision under a name that says time and one that does not, the day-first warning) and classes every difference: `a`, refused outside the list (DT-15, approved by the owner's T1.25b answer); `b.order_open`, `b.order_decided`, `b.format_guess`, `b.offsets`, `b.range`, `b.order_rule` and their mixtures, values in the list read, refused or warned of otherwise (DT-16, proposed for the owner); `c`, inference's text outside the client's shapes under a name that does not say time (DT-17, proposed). Counts: materialize 1,667 of 13,456 agree, 10,623 `a`, 1,166 `b`; the time column's choice 1,630 agree, 10,623 `a`, 1,203 `b`; inference 7,236 of 26,908 agree, 18,815 `a`, 398 `b` (41 of them `b.order_open`), 459 `c`; the warning 2,438 of 2,810 agree, 372 `b.order_rule`. A class `a` answer is made by a value outside the list in both orders: the classifier asserts one for each (`record.column` says `outside`), and `the_differential_reports_the_core_as_it_reads` holds every example's core answer to the core, each reference answer to differ from it, every `a` example to such a value (`made_outside`), and the counts to add up; `an_open_order_in_the_list_is_not_a_refusal_outside_it` is the negative control (`03/15/99`, `15/11/00`: `text_time_order`, never `a`), as is the classifier's own self-check. The old file's 2 MB of pandas readings are gone; `text_times_differential.py` is a protected path beside the recorder.
231+
- Inference: `infer::TextTimes` has `share` (fewest, the values read in either order, most, and how many values lie outside the client's shapes) in place of `never_read`; `text_holds_dates` refuses `text_time_order` where the values read in either order decide the score otherwise than the fewest (an order left open, DT-16), else decides by the score at fewest and at most, the most counting every value outside the list under a name that says time (`names::says_time`), and refuses `text_time` where the two differ (DT-15); `sdm-schema`'s `tests/time_choice.rs` holds the score's edges, the refusal where values could make it, codes under each kind of name, and offsets mixed with a stand-in reader. The time column's choice reads to UTC and refuses only where it asks where a column starts, as before, an open order under `text_time_order` (`TextUnread::Order`, fix round 1). `DATES_READ_DAY_FIRST` names the first day-first value's format where the client's rule decides the day comes first (`Reader::day_first_format`, `assemble::day_first`).
232232
- `materialize`: `Times` reads a fetched text column by the provider's `text_times` (DT-7, unchanged, contract version 16), refusing through `text_unread`: `text_time` (a value outside the list), `text_time_order`, `text_time_zones`; a blank value is dropped and warned of (`TIME_VALUES_UNREAD`). `sdm-core`'s `tests/materialize.rs` refuses each by name and holds the blank's warning.
233233
- No pandas-derived rule remains: `grep -rnE "WORDS|unknown_word|cannot_be_a_date|never_read|fn guess\b|guess_datetime|leap|strptime|dateutil|MISSING: \[" rust/crates/sdm-engine/src/time rust/crates/sdm-schema/src/infer.rs rust/crates/sdm-core/src/materialize.rs` finds the Gregorian leap year of civil arithmetic and the module comment saying no rule follows pandas or `dateutil`, nothing else. H21's question is answered as built (E68) and `dependencies.md`'s row is the core's own spec over `jiff`.
234234
- The client's pins (`tests/test_the_schema_route.py`): two DuckDB system views the core refused (`text_time`, P1B-2) are described as the reference describes them (columns, keys, time column), the one-table `duckdb_tables` meets `column_type` (P1B-10) as the shop's does, and an offset of a day or more or with minutes past 59 is refused by name (DT-15) where the reference reads one or none.
235-
- The differential through the public calls (E49): 60 generated DuckDB sources, 30 text formats each as the orders' time column and as a users feature, on the old route and the core: 24 as the reference, 10 the source's CAST failing (DT-12), 13 refused `text_time` outside the list (DT-15), 13 otherwise in the list (DT-16); 22 changed against T1.25's core. The T0.6 corpus's 121 column cases as source columns under a name that says time and one that does not (242): 60 described as the reference describes them, 36 decided no timestamp as the reference decides and then refused `multicategorical`, 98 refused `text_time` (each column holding a value outside the list that could make the score), 14 `text_time_zones` (6 where the reference raises, DT-11), 34 otherwise in the list (DT-16).
235+
- The differential through the public calls (E49): 60 generated DuckDB sources, 30 text formats each as the orders' time column and as a users feature, on the old route and the core: 24 as the reference, 10 the source's CAST failing (DT-12), 13 refused `text_time` outside the list (DT-15), 13 otherwise in the list (DT-16); 22 changed against T1.25's core. The T0.6 corpus's 121 column cases as source columns under two names, `happened_at` and `label`, neither of which says time to the score (242): 60 described as the reference describes them, 36 decided no timestamp as the reference decides and then refused `multicategorical`, 98 refused `text_time` (each column holding a value outside the list that could make the score), 14 `text_time_zones` (6 where the reference raises, DT-11), 34 otherwise in the list (DT-16).
236236
- Reach: every refusal map, rewritten from both runners (`refusal_map.py write` with `sdm-testkit`'s `tests/refusal_map.rs` observations), is unchanged: the time family's cases reach what T1.25 recorded (the in-format ISO and declared text time columns reach `predict`'s result; the US dates and the unreadable value meet the source's CAST, DT-12), and the AdventureWorks journey reaches `predict`'s result on both hosts without `inference_config`, its codes, names and UUIDs no timestamp.
237237
- The gate: no baselined outcome changes, so no register transition is needed; no contract case is affected by DT-16 or DT-17, so none waits for admission.
238+
- Fix round 1 (review of 90a4202): an inference refusal of values in the list whose order of day and month is open was named `text_time` and counted as class `a`, approved under DT-15; it is `text_time_order` (`Share::listed`, the values read in either order, against `fewest`), in inference and the time column's choice as in `materialize`, and class `b.order_open` under DT-16 (41 inference decisions moved); class `a` needs a value outside the list in both orders, asserted by the classifier and the Rust check. The year written is held to 1 to 9999 before an offset moves the instant (`0000-12-31T23:00-02:00` was read as 0001-01-01T01:00Z), in the reader and the recorder; the vectors hold it and its 9999 mirror (`9999-12-31T23:00-02:00`) with their neighbours inside the list, and the client's `test_text_the_list_does_not_decide_is_refused_under_its_rule` and `test_a_year_the_list_holds_after_an_offset_reads` hold them through `Schema.from_source`. The 11 day-first warning differences DT-17 claimed are in-list order-rule differences, DT-16. Two more seeded faults fail the vectors (the year written unchecked: 25 differ; the values read in either order counted in one: 184); the reader still equals the spec's Python reading on all 13,456 old columns; the probes' counts are unchanged.
238239
- Carried: T1.18's floor-cell waiver (E65(c)) is removed from `ci.yml`, dead since T1.18's landing; `tests/test_contract_ci_gate.py`'s `test_every_cell_gates_a_failing_comparator` is the control, the waiver's own case failing the step in all three cells. T1.18's tracker row says E66(f) answered T1.15's cut.
239240

240241
## T1.26a

0 commit comments

Comments
 (0)