Model Comparisons

The following table presents estimates of the carbon footprint generated by the training popular models on different hardware.

Carbon Footprint and Compute Cost of Training of Popular Models

Model

Hardware

Power (W)

Training Time (hr)

CO₂eq (lbs)

Cloud Compute Cost ($)

Transformerbase

P100x8

1,415.78

12

26

$41 - $410

Transformerbig

P100x8

1,515.43

84

192

$289 - $981

ELMo

P100x3

517.66

336

262

$433 - $1,472

BERTbase

V100x64

12,041.51

79

1,438

$3751 - $12,571

NAS

P100x8

1,515.43

274,120

626,155

$942,973–$3,201,722

Comparisons

Below are some examples of estimate amounts of carbon emitted by standard daily activities. This is based on average rates of carbon emission of air and car travel and lifestyle.

Carbon Footprint of Daily Activities

Consumption

CO₂eq (lbs)

Air travel, 1 passenger, NY↔SF

1,984

Avg Human Life, 1 year

11,023

Avg American Life, 1 year

36,156

Avg Car Including Fuel, 1 lifetime

126,000