2i2c | James Colliander

Digital public goods for Earth system management: U.S. Greenhouse Gas Center launches

Tue, 05 Dec 2023 17:01:48 +0000

(This post first appeared on the 2i2c blog.)

Abstract

The International Interactive Computing Collaboration (2i2c.org), working with NASA VEDA, Development Seed and other partners, operates an interactive computing platform for The U.S. Greenhouse Gas Center. The U.S. GHG Center, announced yesterday at the 28th annual United Nations Climate Conference (COP-28) in Dubai, is an interagency collaboration of the Environmental Protection Agency (EPA), the National Aeronautics and Space Administration (NASA), the National Institute of Standards and Technology (NIST), and the National Ocean and Atmospheric Administration (NOAA). This note places the launch of the U.S. GHG Center in a scientific, international, and national context and argues that similar digital public goods are needed for humanity to understand and manage the Earth system.

Scientific Context

It was controversial in 1827 when Joseph Fourier (the discoverer of the law of heat conduction) argued ¹ that the atmosphere keeps the Earth warm, like a puffy down comforter, but it’s not now. Gases in the atmosphere trap heat near Earth. How much heat is trapped depends on the gas mixture. Putting more heat-trapping gases in is like putting a wool blanket on top of the down comforter. Human activity since industrialization is injecting lots more heat-trapping gas into the atmosphere and changing the Earth’s climate.

The science is clear. The up-to-date consensus view of the global scientific community is expressed in the Sixth Assessment Report of the Intergovernmental Panel on Climate Change (IPCC):

International Context

The international community officially recognized human-influenced climate change at the World Climate Conference (WCC-1) ² in 1979. The 1979 declarationis remarkably prescient and detailed. A complex and interconnected collection of scientific and diplomatic activities were catalyzed by WCC-1. Some important milestones from this history are captured in the chart and numbered list below.

gantt
dateFormat YYYY-MM-DD
title International Climate Change Milestones
section Study
WCC-1 (1979) :crit, done, admin0, 1979-02-12, 1979-02-23
WCP :crit, adminA, 1979-06-01, 2025-12-31
CMIP1: crit, done, adminT, 1995-01-01,1995-12-31
CMIP2: crit, done, adminR, 1997-01-01, 1998-12-31
CMIP2+: crit,done,adminS, 2000-05-09, 2001-12-31
CMIP3 :crit, done, adminP, 2004-10-01, 2006-12-31
CMIP5 :crit, done, adminO, 2008-09-01, 2013-03-15
CMIP6 :crit, done, adminQ, 2014-02-01, 2024-12-31
IPCC :crit, admin1, 1988-12-06, 2025-12-31
IPCC-AR1 :crit, done, adminH, 1990-08-01, 1992-06-30
IPCC-AR2 :crit, done, adminI, 1995-01-01,1995-12-31
IPCC-AR3 :crit,done, adminJ, 2001-01-01, 2001-12-31
IPCC-AR4 :crit, done,adminK, 2007-01-01,2007-12-31
IPCC awarded Nobel Prize :crit, done, adminN, 2007-10-12, 2007-11-12
IPCC-AR5 :crit,done,adminL, 2014-01-01,2014-12-31
IPCC-AR6 :crit,done,adminM,2023-01-01,2023-12-31
section Treaties
Rio Earth Summit (1992) :crit, done, adminC, 1992-06-03, 1992-06-14
UNFCC :crit, admin2, 1994-03-21, 2025-12-31
Berlin (COP-1) :crit, done, adminE, 1995-03-28, 1995-04-07
Byrd-Hagel Resolution :crit, done, adminX, 1997-07-25, 1997-07-30
Kyoto (COP-3) :crit, done, adminD, 1997-12-01, 1997-12-10
Kyoto Protocol :crit, done, admin3, 1997-12-11, 2020-12-31
Paris (COP-21) :crit, done, adminF, 2015-11-30, 2015-12-12
Paris Agreement :crit, adminG, 2016-11-04, 2025-12-31
Glasgow (COP-26) :crit, done, adminV, 2021-10-31, 2021-11-12
Dubai (COP-28) :crit, adminW, 2023-11-20, 2023-12-12
ETF :crit, adminU, 2024-01-01, 2025-12-31

The table above describes a subset (for a more systematic review see ³, ⁴) of key milestones in global efforts to understand and address climate change. A glossary of acronyms and additional background:

The First World Climate Conference (WCC-1) ² was held in 1979.
The World Climate Programme (WCP), an activity overseen by the World Meteorological Organization was established after WCC-1. WCP, in partnership with other organizations, operates programs (e.g. the World Climate Research Program (WCRP)) that organize and integrate international scientific efforts to understand the climate. The WMO also operates the Integrated Global Greenhouse Gas Information System (IG3IS), a natural partner for the emerging work described below.
WCRP manages the Common Model Intercomparison Project (CMIP). CMIP serves as a kind of league for intercomparing models of the Earth’s climate system developed by teams who approach the problems with different methods and designs. Intercomparison, an approach that enables finding the consensus views of teams with divergent approaches to problems, is used in other modeling scenarios.
Research papers on the climate are rapidly produced by scholars from essentially all knowledge disciplines. This overwhelming stream of content, like snowflakes in a blizzard, is coalesced into coherent and carefully scrutinized IPCC Assessment Reports by the Intergovernmental Panel on Climate Change (IPCC).
The Earth Summit held in Rio de Janeiro in 1992 led to the United Nations Framework Convention on Climate Change (UNFCC). The UNFCCC is an international treaty that recognizes the dangers to the climate system caused by human activity, calls for ongoing study, and establishes recurring Conference of the Parties (COP) meetings.
COP-3 (Kyoto) led to the Kyoto Protocol Treaty.
The Byrd-Hagel Resolution was a unanimous United States Senate Resolution that stipulated the United States would not sign treaties promising greenhouse gas reductions by developed countries that did not mandate similar reductions from developing countries. This killed USA participation in the Kyoto Protocol Treaty.
The Paris Agreement, established at COP-21 (Paris), effectively replaces the Kyoto Protocol, includes specifications by participant countries on greenhouse gas reductions called National Determined Contributions (NDCs). The United States entered the Paris Agreement under President Obama, exited under President Trump and rejoined under President Biden.
COP-26 (Glasgow)established an accountability system for the Paris Agreement called the Enhanced Transparency Framework (ETF). Participant countries to the Paris Agreement will submit their first Biennial Transparency Reports (BTR1) under the ETF in 2024.

🇺🇸 U.S.A. Context

The United States Environmental Protection Agency (EPA) annually releases the Inventory of U.S. Greenhouse Gas Emissions and Sinks reports. These reports are submitted to the United Nations in accordance with the UNFCCC. The EPA openly shares software, tools, data, and builds capacity to assist other nations to assemble their own greenhouse gas inventories.

An Interagency Working Group (IWG) on the Social Cost of Carbon was set up by the Obama Administration in 2010. The IWG, renamed as the Interagency Working Group on Social Cost of Greenhouse Gases in 2016, synthesizes research on integrated assessment modelling to quantify the dollar costs associated to damage caused by an incremental increase in GHG emissions in a given year. Quantifying the impacts of GHG emissions in monetary terms is vital to effective rulemaking across the Federal Government. This EPA fact sheet on the social costs of carbon provides further background. A 2017 consensus report ⁵ of the National Academies of Science Engineering and Medicine (NASEM) offered recommendations for ongoing research to improve the assignment of social costs to GHG emissions.

Other federal agencies have developed expertise, data, and analyses that give insights into GHG emissions that compliment and potentially extend the Inventory reports developed annually by the EPA. How should the United States integrate federal agency efforts to monitor and measure greenhouse gas emissions? A 2022 NASEM consensus report ⁶ investigated this question. In January of this year, the Biden Administration released guidance⁷ for federal agencies on incorporating GHG emissions information in policies and reports. Shortly thereafter, a draft federal strategy to advance an integrated greenhouse gas monitoring system was released by NASA with a request for public input.

Some important insights from the NASEM consensus report, the draft federal strategy, and the IWG reports:

Data streams on greenhouse gas emissions can be sorted into two broad categories:
- Activity-based: Activity-based data, sometimes called “bottom-up” data, quantify GHG emissions by measuring activities that generate GHG emissions. Economic or business activity data (gallons of diesel sold in a county on Tuesday; miles flown by a 747 in October) can be converted into quantified GHG emissions information.
- Atmospheric-based: Atmosphere-based data, sometimes called “top-down” data, quantify GHG emissions by performing atmospheric measurements. For example, the Orbiting Carbon Observatory (OCO2) remotely senses $CO_2$ from space.
- A hybrid approach that blends activity-based and atmospheric-based GHG data has the potential to provide new insights.
Interoperable and easily accessed tools and data products for analyzing GHG emissions information and assigning costs should be made available across the Federal government.
GHG emissions information is needed in scenarios outside the international context of participating nations reporting for UNFCCC and Paris Agreement ETF compliance:
- Subnational governments – cities, states, provinces, counties – want GHG data products to measure their progress on emission reduction programs.
- Facilities – harbours, toll roads, power plants, factories, universities – similarly want GHG data products.
- Companies – airlines, trucking, construction – want GHG data products.
Reliable and transparent GHG emissions information is required to enable effective environmental-social-governance (ESG) investment without “greenwashing”.
Accurate and improving quantifications of the social costs associated to GHG emissions require ongoing research.

The U.S. Greenhouse Gas Center

The launch of U.S. GHG Center, an interagency collaboration of the Environmental Protection Agency (EPA), the National Aeronautics and Space Administration (NASA), the National Institute of Standards and Technology (NIST), and the National Ocean and Atmospheric Administration (NOAA), was announced on 2023-12-04 at COP-28 (Dubai)

How much GHG emission is generated through oil and gas production? How much GHG emission is generated by urban centers? Do landfills contribute significant GHG emissions? How do human-generated GHG emissions compare to natural sources of GHG emissions? The U.S. GHG Center is designed to assemble the data, tools, and people to scientifically address these kinds of questions.

The U.S. Greenhouse Gas Center builds on EPA’s leadership to openly share the data and tools for the Inventory and the 2023 Year of Open Science. Instead of building a walled garden with proprietary technology from a vendor selected through RFP, the Center launched a prototype platform using curated open source tools integrated with public federal data. This open toolchain approach amplifies the open data efforts developed over the past two decades.

The U.S. GHG Center’s interactive computing platform is open source science infrastructure. The platform is:

transparently operated by 2i2c on a cloud data center under a right to replicate that ensures zero vendor lock-in with data integrations and visualizations built with using software from a vibrant open source ecosystem by Development Seed, NASA VEDA, and collaborators;
proximate to and optimized ⁸ for analyzing geospatial data (e.g. NASA, NOAA);
designed to be a digital public good.

Research, data and recommendations developed by scientists and engineers that influence policies set by democratic governments should be accessible by voters. No entity should own the ways humans communicate and learn about the Earth system. The U.S. Greenhouse Gas Center’s generous approach to digital infrastructure – an open toolchain adjacent to open data – is vital for democracy and should be replicated across government agencies worldwide.

References

Mémoire sur les températures du globe terrestre et des espaces planétaires. in Oeuvres de Fourier: Publiées par les soins de Gaston Darboux (eds. Fourier, J. B. J. & Darboux, J. G.) vol. 2 95–126 (Cambridge University Press, 2013). ↩︎
World Climate Conference. World Climate Conference - Declaration and supporting documents. https://library.wmo.int/records/item/54699-world-climate-conference-declaration-and-supporting-documents (1979). ↩︎
Gupta, J. A history of international climate change policy. WIREs Climate Change 1, 636–653 (2010). ↩︎
Zillman, J. A history of climate activities. WMO Bulletin 58, (2009). ↩︎
Valuing Climate Changes: Updating Estimation of the Social Cost of Carbon Dioxide. (National Academies Press, 2017). doi:10.17226/24651. ↩︎
Greenhouse Gas Emissions Information for Decision Making: A Framework Going Forward. (National Academies Press, 2022). doi:10.17226/26641. ↩︎
CEQguidance. National Environmental Policy Act Guidance on Consideration of Greenhouse Gas Emissions and Climate Change. Federal Register https://www.federalregister.gov/documents/2023/01/09/2023-00158/national-environmental-policy-act-guidance-on-consideration-of-greenhouse-gas-emissions-and-climate (2023). ↩︎
Abernathey, R. P. et al. Cloud-Native Repositories for Big Scientific Data. Computing in Science & Engineering 23, 26–35 (2021). ↩︎

2i2c + Berkeley

Thu, 21 Oct 2021 09:00:19 +0000

Open Toolchain

Fri, 27 Aug 2021 10:00:00 +0000

Open Toolchain

Fri, 27 Aug 2021 00:00:00 +0000

`Open Toolchain`

Canadian Open Data Society Board meeting

2021-08-27

Idea

People, with the right data, tools and resources, can do good things working together.

James Colliander

Professor of Mathematics: UBC
Cofounder: 2i2c, Callysto, Syzygy, Crowdmark
Past Director (2016-2021): PIMS

Background

PIMS: Distributed Mathematical Sciences Institute

Inspirations

Jessica Hamrick’s Post

Data 8 at Berkeley

Partners

Idea: Jupyter for 🇨🇦

Development History

gantt
title Timeline
section Syzygy
23 universities, 10 research teams: 2020-02-25, 25d
40,000 users : 2021-08-15, 25d
25,000 users : 2020-01-01, 25d
16,000 users : 2019-08-22, 25d
8,000 users : 2017-08-24, 25d
Syzygy service announcement : 2017-03-15, 25d
PIMS + Compute Canada Agreement : 2016-06-02, 25d
Compute Canada Discussions : 2015-12-03, 2016-06-01
Jessica Hamrick's Post : 2015-03-24, 25d
JupyterHub for 🇨🇦 idea : 2015-02-01, 90d
section Callysto
CanCode 2.0 🇨🇦 $1.5M : 2019-10-15, 25d
Callysto content creation : 2018-05-01, 2018-08-31
CanCode 1.0 🇨🇦 $1.5M : 2018-01-30, 25d
Callysto Launched: 2017-10-01, 25d

Examples

Lessons Learned

Research and education transforming
Demand can be met with cloud strategy
Scaling requires infrastructure

Sustainable business model?

`2i2c`

International Interactive Computing Collaboration

Syzygy, Callysto and 2i2c

Tue, 25 Feb 2020 23:00:00 +0000

Communities of practice for Jupyter deployments on shared infrastructure

Thu, 07 Nov 2019 14:00:27 +0000

National Scale Interactive Computing

Wed, 22 Aug 2018 19:32:16 +0000

(Invited post on Jupyter blog: https://blog.jupyter.org/national-scale-interactive-computing-2c104455e062)

This is an invited post from Jim Colliander, Professor of Mathematics at UBC and Director of the Pacific Institute for the Mathematical Sciences.¹

In 2017, the Pacific Institute for the Mathematical Sciences (PIMS), in partnership with Compute Canada and Cybera, launched Syzygy, a cloud-hosted interactive computing platform that delivers JupyterHub deployments for universities across Canada.

Syzygy has been used by over 16,000 students at 20 universities. The main results of the Syzygy experiment so far are:

Demand for interactive computing is ubiquitous² and growing strongly at universities.
The Jupyter ecosystem is an effective way to deliver interactive computing.
A scalable, sustainable, and cost-effective interactive computing service for universities is needed as soon as possible.

Demand for interactive computing

Both research and teaching at universities are adapting to major societal changes driven by explosions in data and computational tools. New educational programs that prepare students to think computationally are emerging, while research strategies are changing in ways that are more open, reproducible, collaborative, and interdisciplinary. These transformations are inextricably linked and are accelerating demand for interactive computing. The Syzygy experiment has shown that using Jupyter in educational programs drives interest in using Jupyter for research (and vice versa). For example, students in mathematics, statistics, and computer science collaborated with a researcher from St. Paul’s Hospital in Vancouver using Syzygy to identify new pathways to prevent death from sepsis. Research communities typically need access to deeper computational resources and often span multiple universities, but the common thread is the need to expand access to interactive computing.

A map of JupyterHub deployments deployed by Syzygy.

Technical milestone achieved

The Syzygy project has demonstrated that it’s possible to deploy tools for interactive computation at a national scale rapidly and efficiently using an entirely open source technology stack. Students, faculty and staff across Canada use Syzygy to access Jupyter through their browsers with their university single-sign-on credentials. The JupyterHubs range from a “standard” configuration to bespoke environments with specially curated tools and data integrations. This richness is possible because of the architecture of the Syzygy and Jupyter projects and the flexibility of the underlying cloud resources. As a case-study, Syzygy demonstrates that the Jupyter community has achieved a significant technical milestone: interactive computing can be delivered at national scale using cloud technologies.

Service level requirements

The validation that interactive computing can be technically delivered at national scale prompts universities to ask a variety of questions. Can interactive computing service be delivered robustly? How will users be supported? What are the uptime expectations? What is the data security policy? How is privacy protected? Can the robustness of the service be clarified in a service level agreement? Syzygy, as an experimental service offered to universities at no charge and without a service level agreement, does not properly address these questions. To advance on their education-research-service mission and address growing demand, universities need a reliable interactive computing service with a service level agreement.

What’s next?

How should universities address their needs for interactive computing over the next five years? Right now, universities are following two primary approaches:

🙏 Ad hoc: faculty figure out how to meet their own needs for interactive computing; IT staff deploys JupyterHub on local or commercial cloud servers; this approach gives universities control over their deployments and hardware, though requires time and expertise that many may not have.
🎩 Use a cloud provider’s service: Google Colab, Amazon Sagemaker, Microsoft Azure Notebooks, IBM Watson Studio; this approach allows universities to quickly launch interactive computing services, with a loss of flexibility and some risks by becoming reliant upon a particular vendor’s closed-source and proprietary software.

These approaches are not sustainable over the long term. If universities all deploy their own JupyterHub services, many will need technical expertise they do not currently have and will involve a significant duplication of effort. If universities rely on hosted cloud notebook services, the reliance on proprietary technology will impair their ability to switch between different cloud vendors, change hardware, customize software, etc. Vendor lock-in will limit the ability of universities to respond to changes in price for the service. Universities will lose agility in responding to changes in faculty, staff, and student computing needs.

There is a third option that addresses the issues with these two approaches and generates other benefits for universities:

🤔 Form an interactive computing consortium: universities collaborate to build an interactive computing service provider aligned with their missions to better serve their students, facilitate research, and avoid risks associated with vendor lock-in.

To retain control over their interactive computing stacks, avoid dependence⁴ on cloud providers, and accelerate the emergence of new programs, universities should work together to deploy interactive computing environments in a vendor-agnostic manner. This might take the form of a consortium — an organization dedicated to serving the needs of universities through customized shared infrastructure for interactive computing. The consortium would also ensure that universities will continue to play a leadership role in the development of the interactive computing tools used for education and research.

The Syzygy experiment confirmed that growing demand for interactive computation within universities can be supplied with the available technologies advanced by the Jupyter open source community. In the coming year, we aim to build upon the success of the Syzygy experiment and seed an initial node of a consortium in Canada with the intention of fostering a global network of people invested in advanced interactive computing. If you are interesting in partnering, please get in touch! See this Jupyter Community Forum post to continue the discussion.

The author gratefully acknowledges feedback on this piece from Ian Allison, Lindsey Heagy, Chris Holdgraf, Fernando Perez, and Lindsay Sill.
Interactive computing needs have been identified in agriculture, applied mathematics, astronomy, chemistry, climate science, computer science, data science, digital humanities, ecology, economics, engineering, genomics, geoscience, health sciences, K-12 education, neuroscience, political science, physics, pure mathematics, statistics, and sociology.
Relying on commercial cloud vendors to provide the interactive computing service for universities risks recreating the problems associated with scientific publishing that emerged with the internet.

2i2c | James Colliander

Digital public goods for Earth system management: U.S. Greenhouse Gas Center launches

Scientific Context

International Context

🇺🇸 U.S.A. Context

The U.S. Greenhouse Gas Center

Openness: Amplified Knowledge Sharing

References

2i2c + Berkeley

Open Toolchain

Open Toolchain

Open Toolchain

Idea

James Colliander

Background

PIMS: Distributed Mathematical Sciences Institute

Inspirations

Jessica Hamrick’s Post

Data 8 at Berkeley

Partners

Idea: Jupyter for 🇨🇦

Development History

Examples

Lessons Learned

Sustainable business model?

2i2c

Syzygy, Callysto and 2i2c

Communities of practice for Jupyter deployments on shared infrastructure

National Scale Interactive Computing

Demand for interactive computing

Technical milestone achieved

Service level requirements

What’s next?

`Open Toolchain`

`2i2c`