NorviTech / Consulting / Automating DA2 extracts
Automating scheduled DA2 extracts into a dashboard
A Discern Analytics 2.0 report that runs on a schedule is only half a pipeline. The other half decides whether the dashboard can be trusted.
The short version
A DA2 report can be scheduled to run in the background and write its output to a file. Point a small script at that file, reshape it, and push it to Power BI, Tableau, Google Sheets or a warehouse, and you have a dashboard nobody has to refresh by hand.
That part takes an afternoon. What takes longer is the handful of assumptions that are true on the first day and false a month later. These are the ones I have seen break real dashboards.
Five assumptions that break
1. “The newest file is my report”
Scheduled reports tend to share one drop folder. Pick the newest file by modified time and the pipeline works only while your report happens to land last; the day another report’s schedule moves, the dashboard ingests a billing extract and falls over. Choose the file by its content instead: the newest drop whose header carries the columns this report must have. Refuse loudly when none does.
2. “The file’s date is the data’s date”
A file stamped with today’s date usually contains yesterday’s activity, because the report ran overnight over the previous day. Get this wrong and every figure is labelled one day late, which nobody notices until a holiday. Confirm the mapping once against the source, and write it into the code as a rule, not a guess.
3. “The job runs after the file lands”
Drop times move: a schedule change, a slow night, a server patch. A job that only ever reads the latest file couples correctness to the clock. Make ingestion catch up instead: on every run, take every drop newer than the last one processed, oldest first. Then the run time stops mattering, and a missed day heals itself on the next run.
4. “An empty file means something broke”
On a genuinely quiet day, such as a clinic closed at weekends, a scheduled report writes a file with a header and no rows. That is a correct answer, not a failure. Treat header-only drops as real zero days, or the dashboard will cry “stale” every Monday, and a warning that fires every Monday is a warning nobody reads.
5. “If it ran, it worked”
The failure that hurts is the silent one: the schedule stops, the dashboard keeps showing last Tuesday, and it looks fine. Stamp every page with the date of the newest data it contains, not the time the script ran, and show a clear warning when that date falls behind. Count the lag in the days the business actually works, so a closed Sunday is not an outage.
What good looks like
Checks its inputs
Refuses a file missing a required column rather than quietly reading every row as zero.
Heals itself
Processes every unread drop in order, so a late file or a skipped run costs a delay, not a hole.
Tells the truth about age
Shows the day the data describes and warns, in business days, when it stops moving.
Next
- Discern Analytics 2.0 consulting — report builds, scheduled extracts, dashboards and training
- DA2, Discern Explorer and the Reporting Portal — which one a request needs
- Why a DA2 report’s totals come out inflated
- Choosing a DA2 report’s date window, and its time zone
- Getting DA2 data into Google, Azure or your warehouse
- DA2 reporting on the Continuum and CommunityWorks domains
Oracle, Oracle Health, Cerner, Millennium and Discern are trademarks of Oracle and/or its affiliates. NorviTech is independent and is not affiliated with, sponsored by or endorsed by Oracle.