magnet system bid evaluation comparing compliance risk scope and technical proposals

Two suppliers respond to the same magnet-system tender.

Supplier A

Price: USD 42,000
Magnetic field: meets requirement
Delivery: 12 weeks
Response: “Fully compliant”

Supplier B

Price: USD 47,000
Magnetic field: meets requirement
Delivery: 14 weeks
Provides:
- Detailed compliance matrix
- Field performance at the specified working gap
- Cooling requirements
- Interface drawing
- Factory acceptance method
- Training scope
- Warranty boundary
- Clear list of buyer-supplied items

Which offer is better?

If the evaluation is based only on price, Supplier A appears to win.

But what if Supplier A’s quoted field applies at a 10 mm pole gap while the buyer requires 30 mm?

What if cooling water is not included?

What if the quotation excludes the power supply?

What if the cryostat integration has never been reviewed?

What if “training included” means one online introduction rather than on-site commissioning?

And what if no one has defined how the final magnetic-field requirement will be accepted?

This is why magnet system bid evaluation should not be reduced to a comparison of headline specifications and price.

For complex scientific equipment, buyers need to evaluate at least three dimensions:

Compliance — does the offer actually meet the requirement?
Risk — how likely is the supplier to deliver the promised result?
Scope completeness — what important hardware, service, interface, or responsibility is missing from the offer?

Modern procurement frameworks support this broader approach. The World Bank’s Rated Criteria framework explicitly uses weighted non-price criteria to evaluate quality, performance, capacity, methodology, risk management, and other project-specific factors rather than relying on price alone. It also recommends separating mandatory pass/fail requirements from criteria that genuinely differentiate competing bids.

New Zealand Government Procurement similarly distinguishes pass/fail preconditions from qualitative criteria such as technical merit, supplier capability, risk, and value for money, and stresses that value for money does not necessarily mean choosing the lowest price.

For electromagnets, Helmholtz coils, Hall systems, VSM, MOKE, and cryogenic magnetic platforms, that distinction is especially important because the final scientific result depends on how multiple subsystems interact.

1. Start With a Pass/Fail Compliance Gate

Not every tender requirement should receive points.

Some requirements are fundamental.

If a system cannot satisfy them, a high score elsewhere may be irrelevant.

Examples might include:

Required magnetic field
Minimum working gap
Mandatory sample space
Required temperature range
Required electrical safety condition
Mandatory bid documents
Required delivery deadline
A specific regulatory requirement where legally applicable

These should be treated as mandatory compliance gates where appropriate.

New Zealand Government Procurement describes preconditions as prerequisite requirements evaluated on a pass/fail basis and recommends using them only for requirements genuinely essential to the deliverable.

Example

Tender requirement:

Magnetic field ≥1 T at 25 mm pole gap.

Supplier A:

0.8 T at 25 mm.

Supplier B:

1.05 T at 25 mm.

If 1 T is genuinely essential to the experiment, Supplier A should not recover by receiving extra points for:

Lower price
Faster delivery
Better-looking software

The scientific requirement comes first.

2. Do Not Make Every Requirement Mandatory

The opposite mistake is equally dangerous.

A tender may classify everything as mandatory:

Maximum field
Highest possible uniformity
Lowest noise
Largest gap
Smallest footprint
Fastest response
Lowest power
Shortest delivery
Lowest price

This can leave:

No supplier technically compliant

or:

Only one artificially favored architecture

A better structure separates:

Mandatory

The experiment cannot succeed without it.

Rated / Scored

Better performance has value, but several acceptable levels exist.

Preferred

Useful, but not worth rejecting an otherwise strong system.

Optional

Future capability or separately priced functionality.

The World Bank similarly recommends avoiding duplication between mandatory pass/fail requirements and weighted Rated Criteria.

3. Score the Offered Configuration, Not the Supplier’s Brochure

This rule should be non-negotiable.

A supplier may manufacture:

0.5 T systems
1 T systems
2 T systems

But if the quotation includes the 0.5 T model, the bid should be evaluated against:

the quoted 0.5 T configuration.

Not against the company’s entire product portfolio.

Weak Evaluation

“The manufacturer has a 1 T magnet on its website, so this requirement is compliant.”

Correct Evaluation

“The quoted model provides 0.5 T. The 1 T model is not included in the commercial offer.”

The compliance score belongs to what the buyer is purchasing.

4. Force Every Magnetic-Field Claim to Include the Working Gap

For electromagnets, one of the most dangerous evaluation mistakes is comparing only:

Maximum field.

For example:

Supplier	Advertised Field
A	1.2 T
B	1.0 T

Supplier A appears better.

But then the detailed conditions reveal:

Supplier	Field	Pole Gap
A	1.2 T	10 mm
B	1.0 T	30 mm

If the experiment requires a 30 mm cryostat, these offers are not directly comparable.

Evaluation Rule

Score:

Magnetic field at the required installed working gap.

Not:

maximum magnetic field under the supplier’s most favorable geometry.

5. Uniformity Must Be Scored Over the Same Volume

The same problem appears with magnetic-field uniformity.

Supplier A:

Uniformity: 0.01%

Supplier B:

Uniformity: 0.05%

Supplier A appears much stronger.

But perhaps:

Supplier A specifies 0.01% over 1 mm DSV.
Supplier B specifies 0.05% over 20 mm DSV.

Without the measurement region, the numbers cannot be meaningfully compared.

Bid Evaluation Should Normalize

Uniformity definition
Volume or area
Sample-center position
Pole gap
Field magnitude
Measurement method where relevant

The evaluator should compare equivalent conditions.

6. “Comply” Should Never Earn Full Credit Without Evidence

A compliance matrix often looks like this:

Requirement	Supplier Response
1 T field	Comply
±B operation	Comply
Water cooling	Comply
USB control	Comply
1-year warranty	Comply

This is not enough for a high technical score.

A stronger evaluation asks:

What supports the claim?

Evidence may include:

Datasheet
Test report
Drawing
Manual
Previous configuration
Factory test method
Engineering calculation

The World Bank’s Rated Criteria guidance specifically encourages suppliers to provide supporting evidence for statements and claims rather than generic responses.

Better Response

Requirement: ≥1 T at 25 mm.

Status: Comply.

Offered value: ≥1.03 T at 25 mm.

Evidence: Magnet performance curve, Drawing/Spec Section 3.1.

That deserves more confidence than one word:

Comply.

7. Score Compliance and Evidence Separately if Necessary

For complex procurements, a useful model is:

Requirement Compliance

Does the technical solution meet the requirement?

Evidence Confidence

How strong is the supporting evidence?

For example:

Response	Compliance	Evidence Confidence
Proven standard product	High	High
Similar product + calculation	High	Medium
New custom design	Potentially high	Medium
“We can do it”	Unknown	Low

This prevents a highly ambitious proposal from scoring exactly the same as a demonstrated configuration.

8. Create a Clear Compliance Vocabulary

A useful evaluation matrix might use:

Comply
Partially Comply
Deviation
Optional
Not Applicable
To Be Confirmed

Do not allow every supplier to invent its own interpretation.

Comply

The quoted configuration meets the requirement.

Partial Compliance

Some but not all of the requirement is met.

Deviation

The offered configuration differs materially.

Optional

Available at additional cost or outside base scope.

To Be Confirmed

Not yet demonstrated or dependent on unresolved information.

The evaluator can then apply consistent treatment across suppliers.

9. Do Not Automatically Give Zero to Every Deviation

A deviation can mean several things.

Harmful Deviation

Buyer requires 1 T.

Supplier offers 0.5 T.

Neutral Alternative

Buyer specifies mechanical polarity reversal.

Supplier provides electronic ±B reversal with a bipolar power supply.

Beneficial Alternative

Buyer expects manual control.

Supplier provides both manual and automated control.

These should not receive the same treatment.

The evaluation question should be:

Does the deviation reduce, preserve, or improve the required scientific functionality?

10. Require Suppliers to Explain the Impact of Every Deviation

A useful deviation statement contains:

Buyer requirement
Supplier offer
Reason for difference
Technical impact
Commercial impact
Alternative if available

Example

Requested: 50 mm pole gap at 1 T.

Offered: 1 T at 35 mm; 50 mm gap available with approximately 0.7 T.

Reason: Magnetic-circuit limitation at required system size.

Impact: Proposed cryostat fits within 35 mm envelope.

Alternative: Larger magnet available at additional cost.

Now the evaluation committee can make a technical decision.

11. Missing Information Is Itself a Risk Indicator

Suppose a supplier does not state:

Pole gap
Cooling requirement
Field uniformity volume
Sample dimensions
Software scope
Training scope

Do not automatically assume the most favorable interpretation.

An unanswered requirement should be treated as:

an unresolved risk.

This is different from a confirmed deviation.

A confirmed deviation can be evaluated.

Missing information cannot.

12. Create a “Missing Scope” Score

This is especially useful for system-level scientific equipment.

A proposal may appear cheaper because important components are absent.

Examples:

Magnet quoted without power supply
Water-cooled magnet without chiller
Cryostat without vacuum pump
Hall system without sample fixture
MOKE system without optical table
VSM option without temperature controller
Software without required communication license
Equipment without installation

These are not necessarily bad proposals.

They become risky when exclusions are unclear.

Scope Completeness Score

You might score:

5 — Complete and clearly itemized

All critical elements and buyer-supplied items identified.

4 — Minor exclusions

No impact on core functionality.

3 — Several interfaces require clarification

2 — Significant hardware or service scope unclear

1 — Price cannot be meaningfully compared

This turns “missing scope” into something the evaluation panel can see.

13. Price Should Be Normalized to the Same Scope

Supplier A:

USD 30,000

Supplier B:

USD 39,000

Supplier A appears 23% cheaper.

But Supplier B includes:

Power supply
Chiller
Gaussmeter
Computer
Training

Supplier A excludes all of them.

The bids are not yet commercially comparable.

Before Price Scoring

Create a normalization sheet:

Tender-required final configuration

minus

supplier-included scope

equals

missing cost exposure.

Only after that should price become a meaningful scored factor.

14. Evaluate Total Project Cost, Not Only Purchase Price

A magnet-system project may create costs for:

Chiller
Cooling-water installation
Electrical work
Freight
Insurance
Customs
Rigging
Installation
Training
Calibration
Maintenance

New Zealand Government Procurement explicitly treats value for money as broader than lowest price and identifies total cost as part of qualitative procurement evaluation.

The cheapest equipment price can therefore produce the highest installed project cost.

15. Integration Risk Deserves Its Own Score

For system-level products, this can be one of the most important categories.

Ask:

How many suppliers are involved?
Who owns the interfaces?
Has this configuration been integrated before?
Are third-party devices included?
Are commands documented?
Are mechanical interfaces frozen?

Lower Integration Risk

One supplier provides and tests:

Magnet
Power supply
Sample stage
Measurement electronics
Software

Higher Integration Risk

Buyer independently combines:

Supplier A magnet
Supplier B cryostat
Existing Keithley electronics
Internally developed software

The second solution may be excellent.

But the buyer is taking more integration responsibility.

That risk should be consciously scored.

16. Score Interface Definition Quality

For complex systems, look for clarity around:

Mechanical

Pole gap
Mounting
Sample height
Optical axis
Cryostat envelope

Electrical

Current
Voltage
Trigger
Analog signals
Connectors

Software

API
Drivers
Commands
Synchronization

Thermal

Sensor
Heater
Cooling

Vacuum

Flanges
Feedthroughs
Pumping requirements

A bid with well-defined interfaces should generally carry less implementation risk than one built on phrases such as:

“Compatible with customer equipment.”

17. Buyer-Supplied Equipment Must Be Included in Risk Scoring

Suppose the buyer wants to reuse:

Existing cryostat
Existing temperature controller
Existing source meter

That can save significant budget.

But it also creates technical dependencies.

Ask whether the supplier has received:

Exact model number
Drawings
Electrical specifications
Interface protocols

Strong Bid

“Compatibility reviewed against Customer Cryostat Drawing ABC Rev. B.”

Weak Bid

“Your cryostat should be fine.”

These should not score equally.

18. Score Standard vs. Custom Content

Customization is not automatically negative.

But it affects delivery risk.

A practical categorization is:

Proven Standard

Already manufactured and documented.

Standard With Modification

Existing design with limited adaptation.

Engineered-to-Order

Significant design work.

First-of-Kind

New architecture or unproven combination.

The more custom the solution, the more important:

Engineering evidence
Design review
FAT
Schedule margin

become.

19. Customization Should Not Receive an Automatic Penalty

A custom solution may be exactly what the experiment requires.

The risk question is not:

“Is it customized?”

It is:

“Is the customization understood and controlled?”

A supplier presenting:

CAD
Simulation
Defined milestones
Acceptance method

may offer lower practical risk than a “standard product” being pushed into an unsuitable application.

20. Score the Credibility of Performance Claims

Consider three bids.

Bid A

“Field stability: excellent.”

Bid B

“Field stability: 100 ppm/h.”

Bid C

“Field stability: ≤100 ppm/h after 30-minute thermal stabilization, under specified cooling conditions.”

Bid C is easiest to evaluate.

A good evaluation rewards:

Quantified claims
Defined conditions
Testability

—not merely ambitious numbers.

21. Distinguish Guaranteed, Typical, and Expected Values

These words matter.

Guaranteed

Should be suitable for contractual acceptance.

Typical

Representative but not necessarily acceptance limit.

Estimated

Engineering prediction.

Target

Design objective.

If Supplier A offers:

Typical field 1.1 T

and Supplier B offers:

Guaranteed field ≥1.0 T

do not automatically treat Supplier A as stronger.

The contractual commitment is different.

22. Score Acceptance Clarity

A technically strong bid should explain:

How will we prove this system works?

Possible verification methods include:

Factory magnetic-field test
Uniformity mapping
Functional software test
Temperature test
Reference-sample measurement
Mechanical inspection

High Acceptance-Clarity Score

Critical requirements map to specific verification methods.

Low Score

“Testing before shipment.”

with no defined test conditions.

For custom systems, this category can substantially reduce post-delivery disputes.

23. FAT Scope Should Match Tender Risk

A simple standard electromagnet may require:

Current
Field
Polarity
Cooling

A custom cryogenic Hall platform may require:

Magnetic field
Temperature
Vacuum
Contact switching
Reference-sample measurement
Automated ±B
Data export

Do not demand the same FAT burden for every project.

But as integration complexity increases, acceptance planning should become more detailed.

24. Score Whether the Acceptance Conditions Match the Requirement

Requirement:

1 T at 30 mm gap.

Supplier FAT:

1 T at 10 mm gap.

That does not verify the tender requirement.

Requirement:

Temperature stability ±0.02 K at 20 K.

Supplier FAT:

System reaches 20 K.

Again, insufficient.

A strong bid keeps the same conditions through:

Requirement → quotation → FAT.

25. Delivery Risk Should Be More Than “Number of Weeks”

Supplier A:

8 weeks

Supplier B:

12 weeks

Supplier A appears better.

But ask:

Standard or custom?
When does the clock start?
Are major components available?
Are drawings still pending?
Is a third-party cryostat required?
Does delivery include FAT?

New Zealand Government Procurement’s due-diligence rules specifically call for assessing whether a supplier has the capability, expertise, systems, capacity, and realistic ability to deliver what is proposed for the quoted price.

A shorter promise is not automatically a lower schedule risk.

26. Score Supplier Capability Separately From Product Specification

Two suppliers may offer similar technical configurations.

Their delivery capability may differ.

Evaluation factors can include:

Relevant experience
Similar systems delivered
Engineering capability
Production capacity
Quality processes
After-sales support

Government procurement guidance similarly identifies capability, capacity, past performance, relevant experience, systems, and risk management as legitimate qualitative evaluation factors.

This should be applied objectively—not used to exclude new suppliers automatically.

27. Relevant Experience Is Better Than Generic Company Size

A company with thousands of employees may have little experience in:

Cryogenic Hall measurements

while a smaller specialist company may have substantial experience in:

Magnet integration
Low-temperature control
Custom sample fixtures

The question should be:

Can this team deliver this project?

not:

Which bidder is the biggest company?

28. References Should Match the Technical Risk

If the project is standard:

A generic customer list may be sufficient.

If the project involves:

4 K operation
±1 T magnetic reversal
Optical access
Custom in-situ sample reversal

then a relevant previous integration matters more.

Ask for:

Similar field
Similar gap
Similar environment
Similar measurement architecture

rather than any unrelated university installation.

29. Evaluate Software as Part of the System

For automated research platforms, software can determine whether the system is actually usable.

Score:

Included functions
Raw-data access
Parameter control
Data export
Automation
API availability
User-defined sequences

For Hall systems, software may need to coordinate:

Current
Magnetic field
Contact switching
Temperature
Calculations

A system with stronger hardware but inadequate software may create greater total project risk.

30. “USB/Ethernet Available” Is Not Enough

Evaluate what the interface actually allows.

Can users:

Read field?
Set current?
Reverse polarity?
Read temperature?
Change setpoint?
Start acquisition?
Export raw data?

A communication port is not an integration specification.

31. Score Documentation Completeness

For larger procurement, useful pre-delivery documents may include:

Technical specification
Compliance matrix
Interface drawing
Site preparation
Programming guide
Acceptance procedure
User manual

A supplier that clearly defines documentation deliverables reduces uncertainty.

Do not award points merely for sending the largest number of PDFs.

Score whether the documents answer the project questions.

32. Site Preparation Risk Should Be Included

A technically compliant system may still be difficult to install.

Evaluate:

Electrical supply
Cooling
Heat load
Equipment weight
Footprint
Network
Delivery route
Rigging
Vacuum/gas utilities

High-Risk Bid

Important site assumptions appear only after award.

Lower-Risk Bid

Site requirements are disclosed before purchase.

For large magnet and cryogenic systems, this difference matters.

33. Do Not Penalize a Supplier Simply for Disclosing Site Requirements

A supplier says:

“We require 400 V three-phase power and 8 L/min cooling water.”

Another says nothing.

The second supplier has not necessarily offered the easier installation.

It may simply have failed to disclose the requirement.

Evaluation should reward:

clarity

rather than reward:

silence.

34. Score Training Scope, Not the Word “Training”

Supplier A:

Training included.

Supplier B:

One remote session plus one day of on-site operator training for up to five users, covering startup, sample loading, standard measurements, software, data export, safety, and basic troubleshooting.

Supplier B gives the evaluator a deliverable.

Supplier A gives a promise.

These should not receive the same score.

35. Installation and Commissioning Should Be Evaluated Separately

Ask:

Installation

Who assembles and connects the system?

Commissioning

Who confirms proper operation?

Training

Who teaches users?

Acceptance

Who proves contractual performance?

A bid may include one but exclude the others.

This matters especially for international purchases.

36. Warranty Should Be Scored on Scope, Not Duration Alone

Supplier A:

24 months

Supplier B:

12 months

Supplier A appears better.

But then:

Supplier A requires the buyer to pay:

International freight both directions
Engineer travel
Labor

Supplier B includes:

Remote diagnosis
Replacement parts
Defined repair procedure

The warranty comparison is no longer one-dimensional.

Evaluate:

Start date
Parts
Labor
Shipping
On-site support
Exclusions

37. After-Sales Support Is a Lifecycle Risk Factor

A research system may remain operational for many years.

Consider:

Spare parts
Calibration
Software support
Replacement sensors
Technical response
Upgrade path

This should not outweigh core technical performance.

But for systems that laboratories depend on for long-term research, support deserves meaningful weight.

38. Score Future Expansion Only if the Path Is Defined

A supplier may claim:

“Cryogenic-ready.”

Ask:

What remains unchanged?
What will be replaced?
What future gap is assumed?
What field remains at that gap?
Is software already compatible?

High Score

Future interface is defined.

Low Score

“Can upgrade later.”

with no architecture.

Future flexibility should be evidence-based.

39. Missing Scope Should Not Be Hidden Inside the Risk Score

It is useful to keep two separate concepts.

Missing Scope

We know something required is not included.

Risk

We do not know whether something will work.

Example:

Vacuum pump excluded = missing scope.

Compatibility with buyer pump not verified = risk.

Separating them helps the committee decide whether the issue is:

Money

Engineering uncertainty.

40. Create a Scope Boundary Matrix

For complex bids, a simple matrix is powerful:

Item	Supplier	Buyer	Optional	Unclear
Magnet	✓
Power supply	✓
Chiller			✓
Computer		✓
Cryostat	✓
Vacuum pump		✓
Integration				?
Training	✓

The final column is often the most important:

Unclear.

Everything there should be resolved before award.

41. Use Risk Severity, Not Just Risk Count

Ten small risks may matter less than one major risk.

A useful risk model considers:

Probability

How likely is the issue?

Impact

How serious would it be?

You can classify:

Low
Medium
High

Example

Missing USB cable:

Low impact.

Cryostat diameter not confirmed against magnet gap:

High impact.

Do not treat them equally.

42. Score Risk Mitigation, Not Just Risk Identification

A good supplier does not need to claim:

“There are no risks.”

A better supplier may say:

Risk: Customer cryostat dimensions are preliminary.

Mitigation: Freeze interface drawing before magnet machining.

That demonstrates project understanding.

The World Bank’s Rated Criteria framework explicitly includes risk management and encourages suppliers to identify project-specific risks and credible mitigations.

This is highly relevant to custom scientific systems.

43. “No Risks” Can Be a Warning Sign

For a highly customized system involving:

Magnet
Cryostat
Optics
Vacuum
Automation

a bid stating:

“No technical risks.”

may indicate that the interfaces have not been examined deeply enough.

Good engineering is not the absence of risk.

It is:

identified risk + controlled mitigation.

44. Do Not Double-Count the Same Strength

Suppose Supplier A has excellent automation.

Do not award points for:

Software
Automation
Ease of operation
Integration

if all four categories measure essentially the same function.

The World Bank recommends keeping Rated Criteria focused on factors that meaningfully differentiate bids and avoiding unnecessary duplication.

A scorecard should be balanced, not engineered to amplify one favored feature.

45. A Practical Magnet-System Scoring Model

One possible structure for a technically complex project could be:

Mandatory Compliance Gate — Pass/Fail

Critical scientific and legal requirements.

Then, among responsive bids:

Technical Performance — 30%

Field
Gap
Uniformity
Stability
Sample compatibility

Scope Completeness — 15%

Included hardware
Services
Accessories
Clear exclusions

Integration & Interface Risk — 15%

Mechanical
Electrical
Software
Cryogenic/vacuum

Acceptance & Evidence — 10%

Verification
FAT
Test evidence

Supplier Capability — 10%

Relevant experience
Capacity
Past performance

Lifecycle Support — 5%

Training
Warranty
Documentation
Spares

Price / Total Cost — 15%

This is only an illustrative framework.

The correct weights depend on the project.

World Bank guidance similarly says Rated Criteria and weightings should be tailored to project priorities and risks rather than mechanically copied from another procurement.

46. Different Projects Need Different Weightings

Standard Electromagnet Purchase

Price may deserve relatively high weight.

Because:

Interfaces are simple
Technical architecture is proven

Custom Cryogenic MOKE System

Integration and technical risk may deserve much greater weight.

Because:

Optical geometry
Magnet
Cryostat
sample alignment
temperature

are tightly coupled.

The scoring model should reflect where the project can actually fail.

47. Do Not Let Price Score Overwhelm Technical Differences

Suppose:

Supplier A:

Technical score: 95/100
Price: USD 100,000

Supplier B:

Technical score: 65/100
Price: USD 80,000

If price receives overwhelming weight, Supplier B may win even though the technical proposal is substantially weaker.

That may be acceptable for a commodity product.

For a research platform whose failure could invalidate the scientific program, it may not be.

The EU public procurement framework similarly allows award decisions to consider price/cost together with qualitative aspects such as technical merit and functional characteristics rather than requiring lowest-price selection.

48. But Do Not Make Technical Scoring So Subjective That Price Becomes Meaningless

The opposite failure is possible.

If criteria include:

“Excellent quality”
“Best technology”
“Professional design”

without clear scoring rules, technical evaluation becomes subjective.

Better Criterion

“Magnetic field at required 30 mm gap.”

Better Criterion

“Extent to which supplier demonstrates mechanical compatibility with supplied cryostat drawing.”

Better Criterion

“Defined FAT method for critical performance.”

Good criteria can be scored from evidence.

49. Write Scoring Descriptors Before Bids Arrive

Do not decide after seeing bids what:

5 points
3 points
1 point

mean.

Example

Interface Definition

5 points: Complete mechanical/electrical/software interfaces documented and integration responsibility clearly assigned.

3 points: Major interfaces defined, minor clarifications remain.

1 point: Significant interfaces unresolved.

0 points: No meaningful interface definition.

This reduces evaluation bias.

50. Use Moderation Across Technical and Procurement Reviewers

Different reviewers naturally notice different things.

Researcher

May prioritize scientific capability.

Engineer

May identify integration risk.

Procurement

May notice exclusions and commercial terms.

Facilities

May identify installation problems.

New Zealand procurement guidance describes evaluation-panel moderation as a way to discuss strengths and weaknesses and reach a common final score rather than relying solely on isolated individual scores.

For complex scientific equipment, this cross-functional review is particularly useful.

51. Do Not Allow One Reviewer to Score Everything

A procurement officer should not be expected to judge:

whether a VSM sensitivity specification is scientifically credible.

A physicist should not be expected to judge:

whether freight terms create an import liability.

Use expertise where it belongs.

A strong evaluation team may include:

Scientific user
Technical engineer
Procurement
Facilities/IT where relevant

52. Clarifications Should Resolve Ambiguity, Not Rewrite the Bid

During evaluation, buyers may need clarification.

Examples:

Confirm field at required gap
Confirm whether chiller is included
Confirm exact warranty start
Provide sample-holder drawing

Clarification is healthy.

But the tender rules applicable to the procurement should govern whether a bidder may materially alter its proposal after closing.

The evaluation team should preserve fairness and consistency.

53. Track Clarifications in the Final Evaluation Record

Do not let important answers remain only in personal inboxes.

If a supplier confirms:

“Yes, the quoted price includes the bipolar power supply.”

that should be recorded in the evaluation package.

Likewise:

Revised drawing
Clarified field
Accepted deviation
Confirmed training

should become part of the decision trail.

54. Red Flags in Magnet-System Bids

Common warning signs include:

“Fully compliant” with no technical response
Maximum field without gap
Uniformity without volume
Stability without duration
Cryogenic range without architecture
Extremely short custom delivery without explanation
No sample-interface information
No cooling requirement
No acceptance method
No clear buyer/supplier boundary
“Upgradeable” without defined interfaces
Multiple third-party instruments with no integration owner

None automatically proves the bid is bad.

They justify clarification or risk scoring.

55. Strong Signals in a Professional Bid

Positive indicators include:

Exact configuration list
Requirement-by-requirement response
Visible deviations
Quantified performance conditions
Interface drawings
Defined site assumptions
Acceptance method
Clear optional items
Buyer-supplied items listed
Risks and mitigations identified
Document revisions controlled

A strong proposal makes the evaluator’s job easier.

That itself is often evidence of project discipline.

56. Lowest Bid and Lowest-Risk Bid Are Not the Same Thing

Consider:

Bid A — USD 50,000

Missing:

Chiller
Installation
Acceptance procedure

Cryostat interface:

Unconfirmed.

Bid B — USD 56,000

Includes:

Chiller
Installation
FAT
Confirmed cryostat interface

The USD 6,000 difference may not actually represent a premium.

It may represent:

Additional hardware
Additional service
Lower integration risk

This is why raw bid prices should not be compared before scope normalization.

57. Nor Should the Lowest-Risk Bid Automatically Win

Risk reduction costs money.

The buyer still needs proportionality.

If two suppliers are technically strong and one charges:

80% more

for only marginally lower risk, the premium may not be justified.

The objective is:

best balance of performance, completeness, risk, and cost.

The World Bank describes Rated Criteria as a way to combine technical quality and financial evaluation to identify the proposal offering the optimal balance of quality and cost.

58. Example: Evaluating Two Electromagnet Bids

Tender requires:

±1 T
25 mm gap
±0.1% uniformity over 10 mm DSV
Continuous duty
Bipolar control
Computer interface

Supplier A

1.2 T maximum
Gap not stated
“High uniformity”
Water cooling
USB
Price USD 18,000

Supplier B

±1.02 T at 25 mm
±0.08% over 10 mm DSV
Continuous water-cooled duty
Four-quadrant power supply
Ethernet + USB
Price USD 21,000

A simplistic evaluation favors A because:

Higher advertised field
Lower price

A technically normalized evaluation may favor B because:

Correct gap
Correct uniform region
Defined operation
Defined power architecture

Supplier A may still become competitive after clarification.

But it has not yet demonstrated that from the submitted bid.

59. Example: Evaluating a Cryogenic Hall System

Tender requires:

±1 T
10–300 K
van der Pauw
Hall mobility
Automated ±B
Existing customer turbo pump

Supplier states:

Magnetic field compliant
Cryostat included
Hall electronics included

But fails to explain:

Cryostat outer diameter
Final magnet gap
Field at that gap
Vacuum interface to customer pump
Temperature-controller integration

This bid may look technically complete.

In reality, several high-risk system interfaces remain open.

That should affect the integration-risk score.

60. Example: Evaluating a MOKE System

Supplier A offers:

Higher maximum field
Standard MOKE optics

Supplier B offers:

Slightly lower field
Confirmed optical access for customer cryostat
Defined sample working distance
Demonstrated ±B reversal without disturbing sample alignment

Which is better?

If the experiment is cryogenic in-situ MOKE, Supplier B may be much closer to the actual scientific objective.

The highest individual specification does not necessarily create the strongest system.

61. Example: Evaluating a VSM Bid

Tender asks for:

Room-temperature hysteresis
Sensitivity ≤5×10⁻⁵ emu
1.5 T
Specific sample sizes

Supplier A:

Sensitivity 1×10⁻⁵ emu
1.2 T

Supplier B:

Sensitivity 5×10⁻⁵ emu
1.7 T

If 1.5 T is mandatory, Supplier A may fail regardless of its stronger sensitivity.

This illustrates why:

mandatory gate first, weighted differentiation second

is often the cleaner evaluation structure.

62. How Cryomagtech Approaches Formal Magnet-System Proposals

For formal university, industrial, integrator, and tender projects, Cryomagtech can structure proposals around the actual technical and procurement questions rather than only supplying a headline specification.

Depending on the project, the offer may address:

Electromagnet or Helmholtz coil configuration
Magnetic field at specified working gap
Uniform region
Duty cycle
Cooling
Power supply
Field measurement
Hall measurement
VSM
MOKE
Cryogenic integration
Sample interfaces
Software
Installation
Training
Acceptance

👉 Product link placeholder: Cryomagtech Magnet & Field Systems, Hall, VSM, MOKE, and Cryogenic Research Solutions

For formal bid evaluation, it is particularly useful when buyers provide:

Mandatory requirements
Rated requirements
Required compliance format
Known interfaces
Required acceptance conditions

That allows each offer to be evaluated on the same technical basis.

63. A Practical Magnet-System Bid Evaluation Scorecard

A buyer could adapt a matrix such as:

Evaluation Area	Example Weight
Mandatory compliance	Pass / Fail
Core technical performance	25%
Sample/application fit	10%
Scope completeness	10%
Integration/interface risk	15%
Acceptance & evidence	10%
Supplier capability	10%
Installation/training/warranty	5%
Evaluated total cost	15%

Again, these percentages are illustrative.

A standard magnet may justify more price weight.

A first-of-kind cryogenic MOKE system may justify more integration-risk weight.

The scoring model should reflect where failure would hurt the project most.

64. Questions Buyers Should Ask Before Final Scoring

Compliance

Does the exact quoted configuration meet every mandatory requirement?
Are operating conditions identical?

Deviations

Which requirements differ?
What is the scientific impact?

Evidence

Which claims are supported?
Standard, calculated, or unproven?

Scope

What hardware is excluded?
What services are excluded?
What must the buyer provide?

Interfaces

Mechanical?
Electrical?
Software?
Cryogenic?
Vacuum?

Risk

What remains technically unresolved?
Who owns each risk?
What mitigation exists?

Acceptance

How will critical performance be verified?

Supplier

Relevant experience?
Capacity?
Support?

Commercial

Normalized total cost?
Delivery?
Warranty?

If these questions are answered, the final score becomes much more defensible.

65. Key Takeaways

Magnet system bid evaluation should not begin with price ranking.
Start by identifying genuinely mandatory pass/fail requirements.
Do not make every desirable specification mandatory.
Evaluate the exact quoted configuration rather than the supplier’s broader product portfolio.
Magnetic field must be compared at the same working gap.
Uniformity must be compared over the same defined region.
“Comply” is stronger when supported by evidence.
Missing information should be treated as unresolved risk rather than silently assumed compliant.
Technical deviations should be evaluated according to their effect on the scientific objective.
Missing scope should be scored separately from technical risk.
Bid prices should be normalized to equivalent scope before price comparison.
Integration and interface risk deserve substantial attention for Hall, MOKE, VSM, and cryogenic systems.
Supplier capability, evidence, and acceptance planning help distinguish a credible proposal from an ambitious specification sheet.
Site preparation, training, installation, warranty, and documentation are part of project risk.
Future upgradeability should be scored only when interfaces are defined.
Weighted criteria should focus on factors that genuinely differentiate proposals rather than duplicating mandatory requirements.
Value for money does not necessarily mean lowest purchase price.

The weak evaluation question is:

“Which compliant supplier has the lowest price?”

The stronger question is:

“Which responsive bid gives us the best combination of scientific performance, complete scope, controlled interfaces, credible evidence, acceptable delivery risk, clear acceptance responsibility, and evaluated total cost?”

That is a much better basis for purchasing a research magnet system.

References

World Bank — Rated Criteria

The World Bank’s Rated Criteria framework explains the use of weighted non-price factors to evaluate qualities such as performance, methodology, risk management, capability, and other project-specific attributes alongside financial evaluation. It also recommends clearly defining required evidence and weightings in the procurement documents.

World Bank — Rated Criteria

New Zealand Government Procurement — Evaluation Criteria

Official procurement guidance distinguishes pass/fail preconditions from qualitative criteria including technical merit, supplier capability, risk, and value for money, and explains that value for money is broader than simply selecting the lowest price.

New Zealand Government Procurement — Decide Your Evaluation Criteria