Appendix A — Understanding the Service Layer

The chart modules move quickly through the service layer — a call method is introduced, the data comes out shaped correctly, and we move on to the chart. This appendix slows down and examines what the services are actually doing.

The patterns here — group_by, transform_values, each_with_object, aggregation, pivot — appear throughout Rails applications. Understanding them makes the services readable rather than mysterious.

A.1 — The extend self Pattern

Every service in this series uses extend self:

1
2
3
4
5
6
7
8
9

module Stats
  module LabourForce
    extend self

    def call(readings)
      # ...
    end
  end
end

Without extend self, calling Stats::LabourForce.call(readings) would raise NoMethodError — call would be an instance method with no instance to call it on.

extend self makes every method defined in the module into both an instance method and a module-level method. This means call can be invoked directly on the module:

1
2

Stats::LabourForce.call(readings)   # works
Stats::LabourForce.new.call(readings) # also works, but nobody does this

The alternative is a class with def self.call:

1
2
3
4
5

class Stats::LabourForce
  def self.call(readings)
    # ...
  end
end

Both work. extend self on a module is preferred here because it signals that this is procedural code — it takes input and returns output, maintains no state, and has no lifecycle. A class implies an object with identity and state; a module with extend self implies a namespace for functions. The intent is clearer.

A.2 — The Core Transformation Pattern

Most services follow the same shape:

1
2
3
4
5

def call(readings)
  readings
    .group_by { |r| r.some_key }
    .transform_values { |rows| rows.map { |r| shape(r) } }
end

Step 1 — group_by

group_by takes an enumerable and returns a Hash where keys are the return value of the block and values are arrays of elements that produced that key:

1
2
3
4
5
6

readings.group_by(&:industry)
# => {
#      "Mining"      => [<GdpReading industry="Mining" year=2000 quarter=1 ...>, ...],
#      "Agriculture" => [<GdpReading industry="Agriculture" year=2000 ...>, ...],
#      ...
#    }

The &:industry shorthand is equivalent to { |r| r.industry }. Each array contains all readings for that industry across all years and quarters.

Step 2 — transform_values

transform_values maps over the values of a Hash, leaving the keys unchanged:

1
2
3
4
5
6
7
8

.transform_values do |rows|
  rows.map { |r| { year: r.year, quarter: r.quarter, value: r.value_billions.to_f } }
end
# => {
#      "Mining"      => [{ year: 2000, quarter: 1, value: 3.2 }, ...],
#      "Agriculture" => [{ year: 2000, quarter: 1, value: 1.1 }, ...],
#      ...
#    }

Each array of ActiveRecord objects is mapped to an array of plain Ruby hashes. The result is a Hash of Array of Hash — nested but regular, easy to iterate in a chart component.

Why plain hashes?

Returning ActiveRecord objects from a service ties the consumer to the model interface. If a column is renamed, every consumer breaks. Plain hashes with symbolic keys are stable, serialisable, and testable without a database.

A.3 — The BigDecimal.to_f Rule

Every service calls .to_f on decimal database columns:

1

value: r.value_billions.to_f

Without .to_f, Rails returns a BigDecimal object from the database. BigDecimal serialises to a quoted JSON string:

1
2

{ "value": "3.2" }   // BigDecimal — quoted string
{ "value": 3.2 }     // Float — unquoted number

ECharts expects unquoted numbers. Quoted strings are silently treated as zero or cause incorrect rendering. .to_f converts BigDecimal to a plain Ruby Float which serialises to an unquoted JSON number.

This is a consistent rule: always call .to_f on any decimal column before including it in a service result.

A.4 — The Labour Force Service: Pivoting Rows

The Labour Force service is more complex than the GDP service because the data arrives in a different shape. Let us dissect it in detail.

The raw data shape

The labour_force_readings table has one row per state per month, with separate columns for each measure:

state   year  month  employed_thousands  unemployed_thousands  participation_rate  unemployment_rate
NSW     2012  1      3421.3              198.4                 62.1                5.5
NSW     2012  2      3418.7              196.1                 62.0                5.4
VIC     2012  1      2891.2              161.3                 64.3                5.3
...

What we want

For charts we want annual averages per state — one hash per year, not one hash per month:

1
2
3
4
5
6
7
8

{
  "New South Wales" => [
    { year: 2012, employed: 3419.8, unemployed: 197.1, participation: 62.1, rate: 5.4 },
    { year: 2013, employed: 3445.2, unemployed: 189.3, participation: 62.3, rate: 5.2 },
    ...
  ],
  ...
}

The transformation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

def call(readings)
  readings
    .group_by(&:state)                          # 1
    .transform_values do |state_rows|
      state_rows
        .group_by(&:year)                       # 2
        .map do |year, rows|                    # 3
          {
            year:          year,
            employed:      avg(rows, :employed_thousands),   # 4
            unemployed:    avg(rows, :unemployed_thousands),
            participation: avg(rows, :participation_rate),
            rate:          avg(rows, :unemployment_rate)
          }
        end
        .sort_by { |r| r[:year] }               # 5
    end
end

Step 1 — group_by(&:state)

Groups all readings by state name. Result:

1
2
3
4
5

{
  "New South Wales" => [156 readings],
  "Victoria"        => [156 readings],
  ...
}

Step 2 — group_by(&:year) inside transform_values

For each state’s readings, groups again by year. This is a nested group_by — perfectly legal in Ruby and a common pattern:

1
2
3
4
5

{
  2012 => [12 readings — Jan through Dec],
  2013 => [12 readings],
  ...
}

Step 3 — map over the year groups

group_by returns a Hash. Calling map on a Hash yields [key, value] pairs — destructured here as |year, rows|. Each iteration produces one hash for one year.

Step 4 — avg helper

1
2
3
4
5

def avg(rows, attr)
  values = rows.map { |r| r.send(attr).to_f }.reject(&:zero?)
  return 0.0 if values.empty?
  (values.sum / values.size).round(1)
end

r.send(attr) calls the method named by attr on each reading — equivalent to r.employed_thousands when attr is :employed_thousands. This avoids writing four separate average calculations.

.reject(&:zero?) excludes zero values from the average. A missing month is stored as zero in the database — including it would drag the annual average down incorrectly. This is a domain decision: treat zero as “no data”, not as “zero employed persons”.

Step 5 — sort_by { |r| r[:year] }

group_by does not guarantee key order. Sorting by year ensures the array is in chronological order before it reaches the chart component.

A.5 — The Business Composition Service: A Richer Example

The Stats::BusinessComposition service is the most interesting in the series. It demonstrates several patterns not seen in the other services.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

module Stats
  module BusinessComposition
    extend self

    def call(readings)
      latest = readings.maximum(:year) * 10 + readings.maximum(:quarter)  # 1

      latest_readings = readings.select { |r|                              # 2
        r.year * 10 + r.quarter == latest
      }.reject { |r|
        r.industry == "Total" || r.sales_billions.to_f.zero?              # 3
      }.sort_by { |r| -r.sales_billions.to_f }                            # 4

      total = latest_readings.sum { |r| r.sales_billions.to_f }           # 5

      latest_readings.map do |r|                                           # 6
        {
          industry: r.industry,
          sales:    r.sales_billions.to_f.round(1),
          share:    ((r.sales_billions.to_f / total) * 100).round(1)
        }
      end
    end
  end
end

Step 1 — Finding the latest quarter

1

latest = readings.maximum(:year) * 10 + readings.maximum(:quarter)

We need to identify the most recent year + quarter combination. The naive approach — readings.order(year: :desc, quarter: :desc).first — works but requires sorting the entire dataset.

The sort key trick is simpler: multiply year by 10 and add quarter. Since quarters are 1–4, this produces a unique integer for each year/quarter combination:

2024 Q1 → 2024 * 10 + 1 = 20241
2024 Q2 → 2024 * 10 + 2 = 20242
2024 Q3 → 2024 * 10 + 3 = 20243
2024 Q4 → 2024 * 10 + 4 = 20244

The maximum of these values corresponds to the latest quarter. Calling readings.maximum(:year) and readings.maximum(:quarter) separately is safe because the latest year always contains the latest quarter — the dataset has no gaps.

Note: This trick works because quarters are 1–4, a single digit. It would fail for months (1–12) — 2024 * 10 + 12 = 20252 is less than 2024 * 10 + 9 = 20249… wait, no: 20252 > 20249 so it actually works for months too. But for clarity, use year * 100 + month for monthly data to make the intent obvious.

Step 2 — Selecting the latest quarter’s rows

1
2
3

latest_readings = readings.select { |r|
  r.year * 10 + r.quarter == latest
}

select filters the array to rows matching the latest quarter. The same sort key applied to each row identifies matches efficiently without string comparison.

Note that readings here is an ActiveRecord relation — calling select with a block loads all records into memory and filters in Ruby. For large tables, a database WHERE clause would be more efficient. Our dataset is small enough that in-memory filtering is fine and keeps the service independent of ActiveRecord query syntax.

Step 3 — Rejecting totals and zeros

1
2
3

.reject { |r|
  r.industry == "Total" || r.sales_billions.to_f.zero?
}

The ABS data includes a “Total” row that sums all industries. Including it in a pie chart would create a slice larger than all others combined — visually nonsensical. We reject it explicitly by name.

We also reject zero values — some industry/quarter combinations have no data reported, stored as zero. Including them produces zero-value slices in the chart.

Step 4 — Sorting descending by sales

1

.sort_by { |r| -r.sales_billions.to_f }

sort_by { |r| -value } sorts descending — the negative sign reverses the natural ascending order. Sorting largest-first means the pie chart renders the dominant industries first, which ECharts places at the top-right of the chart by default.

Step 5 — Computing the total

1

total = latest_readings.sum { |r| r.sales_billions.to_f }

total is computed after filtering — it is the sum of the industries we are actually displaying, not the sum of all industries including “Total”. This ensures percentages add to 100%.

Step 6 — Building the result

1
2
3
4
5
6
7

latest_readings.map do |r|
  {
    industry: r.industry,
    sales:    r.sales_billions.to_f.round(1),
    share:    ((r.sales_billions.to_f / total) * 100).round(1)
  }
end

Each reading becomes a plain hash with three keys. share is the percentage of total — computed here in Ruby rather than left to ECharts. This means the share is available for:

• Custom tooltip formatters (which receive the raw value, not the ECharts-calculated percent)
• Testing in plain Ruby
• Any non-chart consumer of this service (a table, a CSV export, an API response)

ECharts calculates its own percent for label templates ({d}) — but the ECharts percentage may differ slightly from ours due to floating point rounding. Consistent rounding in the service prevents discrepancies between the tooltip and the label.

A.6 — Testing Services

Services are the most testable layer in the stack. No database, no browser, no Rails — just Ruby objects in and Ruby hashes out.

Testing Stats::LabourForce

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

# test/services/stats/labour_force_test.rb
class Stats::LabourForceTest < ActiveSupport::TestCase
  def readings
    [
      LabourForceReading.new(
        state: "New South Wales", year: 2022, month: 1,
        employed_thousands: 4100.0, unemployed_thousands: 160.0,
        participation_rate: 63.2, unemployment_rate: 3.8
      ),
      LabourForceReading.new(
        state: "New South Wales", year: 2022, month: 2,
        employed_thousands: 4120.0, unemployed_thousands: 158.0,
        participation_rate: 63.4, unemployment_rate: 3.7
      ),
      LabourForceReading.new(
        state: "Victoria", year: 2022, month: 1,
        employed_thousands: 3400.0, unemployed_thousands: 140.0,
        participation_rate: 65.1, unemployment_rate: 4.0
      )
    ]
  end

  test "groups by state" do
    result = Stats::LabourForce.call(readings)
    assert_equal %w[New\ South\ Wales Victoria], result.keys.sort
  end

  test "averages months within each year" do
    result = Stats::LabourForce.call(readings)
    nsw    = result["New South Wales"].first

    assert_equal 2022,   nsw[:year]
    assert_equal 4110.0, nsw[:employed]    # average of 4100 and 4120
    assert_equal 3.8,    nsw[:rate]        # average of 3.8 and 3.7, rounded
  end

  test "sorts years chronologically" do
    result = Stats::LabourForce.call(readings)
    years  = result["New South Wales"].map { |r| r[:year] }
    assert_equal years.sort, years
  end

  test "excludes zero values from averages" do
    readings_with_zero = readings + [
      LabourForceReading.new(
        state: "New South Wales", year: 2022, month: 3,
        employed_thousands: 0.0, unemployment_rate: 0.0,
        participation_rate: 0.0, unemployed_thousands: 0.0
      )
    ]
    result = Stats::LabourForce.call(readings_with_zero)
    nsw    = result["New South Wales"].first

    # Zero month should not drag down the average
    assert_equal 4110.0, nsw[:employed]
  end
end

Testing Stats::BusinessComposition

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

# test/services/stats/business_composition_test.rb
class Stats::BusinessCompositionTest < ActiveSupport::TestCase
  def readings
    [
      BusinessIndicatorReading.new(industry: "Mining",         year: 2024, quarter: 3, sales_billions: 142.3),
      BusinessIndicatorReading.new(industry: "Manufacturing",  year: 2024, quarter: 3, sales_billions: 98.7),
      BusinessIndicatorReading.new(industry: "Retail Trade",   year: 2024, quarter: 3, sales_billions: 76.4),
      BusinessIndicatorReading.new(industry: "Total",          year: 2024, quarter: 3, sales_billions: 317.4),
      BusinessIndicatorReading.new(industry: "Mining",         year: 2024, quarter: 2, sales_billions: 138.1),
    ]
  end

  test "returns only the latest quarter" do
    result = Stats::BusinessComposition.call(readings)
    assert result.all? { |r| r[:industry] != "Mining" || result.size < 5 }
    # All results should be from 2024 Q3, not Q2
    assert_equal 3, result.size  # Mining, Manufacturing, Retail (not Total)
  end

  test "excludes Total row" do
    result = Stats::BusinessComposition.call(readings)
    assert_nil result.find { |r| r[:industry] == "Total" }
  end

  test "sorts by sales descending" do
    result = Stats::BusinessComposition.call(readings)
    sales  = result.map { |r| r[:sales] }
    assert_equal sales.sort.reverse, sales
  end

  test "shares add to 100" do
    result = Stats::BusinessComposition.call(readings)
    total  = result.sum { |r| r[:share] }
    assert_in_delta 100.0, total, 0.5  # allow for rounding
  end

  test "includes share percentage" do
    result = Stats::BusinessComposition.call(readings)
    mining = result.find { |r| r[:industry] == "Mining" }

    expected_share = (142.3 / (142.3 + 98.7 + 76.4) * 100).round(1)
    assert_equal expected_share, mining[:share]
  end
end

Run all service tests:

1

rails test test/services/

No database seed required — the tests use Model.new to build objects in memory. This makes the test suite fast and independent of data quality in the seed fixtures.

A.7 — Patterns Summary

A.8 — The National Accounts Gauges Service: Parallel Series and each_with_object

The Stats::NationalAccountsGauges service introduces two new patterns: building a Hash from a flat array using each_with_object, and aligning multiple series on a shared set of periods using Ruby’s array intersection operator.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

module Stats
  module NationalAccountsGauges
    extend self

    def call
      gdp    = series("Gross domestic product")
      saving = series("Household saving ratio")
      trade  = series("Terms of trade")

      periods = gdp.keys & saving.keys & trade.keys    # 1

      {
        periods: periods.map { |p| period_label(p) },  # 2
        gdp:     periods.map { |p| gdp[p] },
        saving:  periods.map { |p| saving[p] },
        trade:   periods.map { |p| trade[p] }
      }
    end

    private

    def series(indicator)
      NationalAccountsReading
        .where(indicator: indicator)
        .order(:year, :quarter)
        .pluck(:year, :quarter, :value)
        .each_with_object({}) do |(year, quarter, value), h|   # 3
          h[[year, quarter]] = value.to_f.round(1)
        end
    end

    def period_label(period)
      "#{period[0]} Q#{period[1]}"
    end
  end
end

Step 3 — each_with_object in detail

This is the most interesting pattern in the service. Let us break it down:

1

.pluck(:year, :quarter, :value)

pluck with multiple columns returns an array of arrays:

1
2
3
4
5
6

[
  [2000, 1, 351882],
  [2000, 2, 355304],
  [2000, 3, 358100],
  ...
]

We want to turn this into a Hash keyed by [year, quarter]:

1
2
3
4
5
6

{
  [2000, 1] => 351.9,
  [2000, 2] => 355.3,
  [2000, 3] => 358.1,
  ...
}

each_with_object({}) is the right tool. It takes an accumulator object — here an empty Hash {} — and passes it alongside each element into the block. The accumulator persists across all iterations and is returned at the end:

1
2
3

.each_with_object({}) do |(year, quarter, value), h|
  h[[year, quarter]] = value.to_f.round(1)
end

The block parameters need explaining. each_with_object yields two things: the current element and the accumulator. Since each element is itself an array [year, quarter, value], Ruby destructures it automatically when you write |(year, quarter, value), h| — the inner parentheses destructure the element, h receives the accumulator.

Without destructuring it would look like:

1
2
3
4

.each_with_object({}) do |row, h|
  year, quarter, value = row
  h[[year, quarter]] = value.to_f.round(1)
end

Both are equivalent — the destructured form is more concise.

Why each_with_object rather than map + to_h?

The map + to_h alternative:

1
2

.map { |year, quarter, value| [[year, quarter], value.to_f.round(1)] }
.to_h

This also works. each_with_object is preferred when the transformation isn’t a simple 1-to-1 mapping — for example if you needed to conditionally add multiple keys, or merge into an existing Hash. For this case either is fine; each_with_object is used here because it makes the intent explicit: we are building a Hash, one entry at a time.

Why a compound key [year, quarter]?

Using [year, quarter] as the Hash key preserves both dimensions without string formatting. The key [2000, 1] is unambiguous and easy to reconstruct. Compare with a string key "2000-Q1" — that works but requires parsing to extract the year and quarter back out. Array keys avoid this round-trip.

Step 1 — Array intersection &

1

periods = gdp.keys & saving.keys & trade.keys

Array#& returns the intersection — elements present in both arrays, in the order they appear in the left operand:

1
2

[1, 2, 3, 4] & [2, 4, 6] & [1, 2, 4]
# => [2, 4]

Applied to Hash keys, it finds the periods present in all three series. If the ABS data has a quarter for GDP but not for saving ratio (which can happen near the current release date when some series update before others), the intersection silently excludes it. No nil values, no array length mismatches.

Step 2 — Building parallel arrays

1
2
3
4
5
6

{
  periods: periods.map { |p| period_label(p) },
  gdp:     periods.map { |p| gdp[p] },
  saving:  periods.map { |p| saving[p] },
  trade:   periods.map { |p| trade[p] }
}

By mapping over the same periods array for each series, we guarantee parallel arrays — gdp[0], saving[0], and trade[0] all correspond to periods[0]. The slider in Module 08 relies on this — index i into any array gives the correct value for that period.

This is a deliberate structural choice. An alternative would be an array of period objects:

1

periods.map { |p| { period: period_label(p), gdp: gdp[p], saving: saving[p], trade: trade[p] } }

That works too and keeps each period’s data together. The parallel arrays approach is chosen here because it maps more directly to what the JavaScript slider needs: separate arrays it can index into independently.

More services will be added to this appendix as the series progresses.