# OBE Calculation Review Against International Practice

Review date: 30 September 2026. Scope: every calculation from mark entry to the Student Profile radar, compared with the practice expected by outcome-based accreditation (Washington Accord signatories such as BAETE, ABET, NBA, EAC Malaysia, EUR-ACE).

## 1. What the standards actually require

None of these frameworks prescribes a single formula. They require that outcome measurement is:

1. **Valid**: a score reflects the evidence the course's own grading scheme says it should, weighted the way the scheme weights it.
2. **Traceable**: assessment item → CO → PO through a documented mapping (correlation 1/2/3 = low/medium/high).
3. **Direct and indirect**: marks, plus a survey or other indirect measure, combined with stated weights.
4. **Judged against stated targets**: a student-level threshold and a cohort-level benchmark, published with every report.
5. **Sufficiently evidenced**: a PO judgement rests on more than one assessment point where possible, and gaps in coverage are visible, not hidden.
6. **Closed-loop**: results that miss the target lead to documented corrective action (CQI).

### The spider (radar) chart in OBE reporting

The radar is used to show one student's (or one cohort's) attainment across all program outcomes at once. Accepted practice:

| Principle | Why it matters |
|---|---|
| One axis per program PO, always the same set and order (PO1–PO12, clockwise) | A radar's shape and area depend on the axes. Dropping or reordering axes makes two students' charts look different even when the scores are the same. |
| Fixed radial scale 0–100 %, starting at 0 | A truncated scale exaggerates differences. |
| An unassessed PO is shown as missing, not as 0 | A 0 looks like failure. "No evidence" is a coverage gap, a different finding. |
| Reference polygons: the target or pass ring, and the cohort average | These let the reader see whether the student is below the standard or below their peers. |
| Evidence strength stated per PO | A PO backed by one course is provisional. Accreditors look for triangulation across courses. |
| PO value = weighted mean of course contributions, weight = credit × mapping strength (or marks mapped) | Big, strongly aligned courses should count more than small, weakly aligned ones. |

## 2. Audit of the OBEsoft pipeline

| Stage | OBEsoft before this review | Reference practice | Verdict |
|---|---|---|---|
| Mark entry: blank vs 0, mandatory and choice questions (R1–R5), absentees | Blank = not attempted, 0 = scored zero; choice questions judged fairly; absentees excluded | Same principle: never penalise a legitimate choice, never reward an unanswered compulsory question | **Meets.** Better than most tools. |
| Question → CO mapping | Each question tagged with one CO; the completion checklist blocks unmapped questions | Same | **Meets** |
| **Student CO score** | Raw marks of all assessments added together; the declared assessment weight (%) was **ignored** | Marks are scaled to the course's weighting scheme before pooling, so the CO score agrees with how the course is graded | **Gap. Fixed:** *Assessment-weighted* basis |
| CO attainment | % of students strictly above the pass mark (40); attained when ≥ cohort benchmark (50 %) | % of students at or above a threshold vs a target; thresholds are institutional policy | **Meets.** "Strictly above" is a documented local policy (AUST workbook parity). |
| CO–PO correlation matrix | 1/2/3 strengths | Same | **Meets** |
| **Student PO score** | Weight = strength ÷ the CO's total strength across all POs (OBE Calculation Aid) | Weight = the correlation strength for that PO: PO = Σ(s·CO) ÷ Σ s | **Gap. Fixed:** *CO–PO strength* weighting. Under the old formula a CO with strength 3 to five POs (weight 3/15 = 0.2) counted less for PO1 than a CO with strength 1 to PO1 only (weight 1/1 = 1.0). |
| Indirect measure (CO exit survey) | Share of responses rated ≥ 4 on a 5-point scale | Same | **Meets** |
| Combined attainment | Direct × 80 % + indirect × 20 %, judged against the same benchmark | 80/20 or 70/30, stated in the report | **Meets** |
| Student Outcome Profile | Weighted mean over courses, weight = credit × Σ strength; best score kept for a retaken course | Same idea (credit and mapping weighted); retake policy is local | **Meets** |
| **Profile radar** | Only the assessed POs had axes, so the chart shape changed from student to student. With fewer than 3 POs it was not a polygon at all. Missing POs were invisible, and evidence strength was not shown. | Fixed axis set, missing ≠ 0, evidence stated | **Gap. Fixed** |
| Bloom domains | The level selector offered C1–C6 even for Affective and Psychomotor COs, so A-levels and P-levels could never be recorded | Krathwohl A1–A5, Simpson P1–P7 | **Gap. Fixed** |
| CQI | Unattained COs listed; root cause and action plan required before completion | Same | **Meets** |

## 3. What changed

1. **CO Score Basis** (per course, stored as `cfg.coBasis`):
   - **Assessment-weighted (standard).** Each mark and maximum is multiplied by the assessment's weight ÷ its full marks. Full marks count the mandatory questions plus the *k* largest choice units.
   - **Raw marks (legacy).**
   - Example: quiz 9/10 (weight 20 %) and final 30/100 (weight 80 %) on the same CO give **50 %** weighted, against 35.5 % raw. The raw figure let the 100-mark final count ten times as much as the quiz, while the course's grading scheme weights it only four times as much (80 % vs 20 %).
2. **CO → PO Weighting** (`cfg.poWeighting`): **CO–PO strength (standard)** or **Normalised per CO (legacy)**.
3. **Safe roll-out:**
   - New courses get the Setup defaults, which are the standard methods.
   - Every existing course keeps the legacy methods, so no published number and no saved profile changes silently.
   - A course can be switched in Course Overview. For a completed course, reopen and complete it again, or run `php bin/rebuild-student-outcomes.php`, to re-freeze its scores.
   - If a CO-mapped assessment has no weight, the course falls back to raw marks and the Attainment tab says which assessment is missing its weight.
4. **Transparency:** the Attainment tab and the printed Attainment Report state the method used, next to the thresholds they already listed.
5. **Radar:**
   - All 12 PO axes appear in a fixed order.
   - A PO with no evidence is shown as "n/a", a gap rather than 0.
   - A PO backed by only one course has a hollow point and is reported as provisional.
   - The PDF and Excel exports list every PO with an Evidence column and add an "Evidence coverage" finding.
6. **Bloom levels:** the domain-aware selector records A1–A5 and P1–P7. The server accepts only valid codes.

Both calculation engines (browser and server) were changed identically. They pass 120 hand-checked fixture checks and agree on 400 random courses.

## 4. Kept as institutional policy (no change needed)

These are acceptable under every framework, provided they are published. OBEsoft prints them on every report.

- Pass line 40 %, KPI marks 50 %, cohort benchmark 50 %, applied as "strictly above". Many programmes use "at or above" and 50–60 %. Adjust in Setup if the programme's Self-Assessment Report says otherwise.
- 80/20 direct/indirect split and a Likert pass rating of 4.
- Best score for a retaken course.

## 5. Recommended next steps (not implemented)

1. **Program-level PO dashboard:** a cohort radar per intake, aggregating all completed courses. This is the chart accreditation visits usually ask for first.
2. **Attainment levels:** optionally report 0–3 levels (e.g. ≥ 60 % of students → 3, 50–59 → 2, 40–49 → 1). Some agencies (NBA) expect them.
3. **Minimum cohort size:** flag CO or PO results from fewer than about 10 students as statistically weak.
4. **Program-level indirect measures:** graduate exit, alumni and employer surveys feeding POs directly, not only through CO exit surveys.
5. **Rubrics for affective and psychomotor COs:** now that A-levels and P-levels can be recorded, assess them with rubrics rather than written questions.
6. **Update `docs/OBEsoft_Attainment_Methodology.docx`** sections 2, 4 and 7 to describe the two methods. The Word guide still describes only the legacy formulas.