What’s New in Astropy 5.0?

Overview

Astropy 5.0 is a major release that adds significant new functionality since the 4.3.x series of releases. In addition, it is a long-term support release (LTS) which will be supported with bug fixes for two years.

In particular, this release includes:

In addition to these major changes, Astropy v5.0 includes a large number of smaller improvements and bug fixes, which are described in the Full Changelog. By the numbers:

  • 1323 commits have been added since 4.3

  • 146 issues have been closed since 4.3

  • 258 pull requests have been merged since 4.3

  • 54 people have contributed since 4.3

  • 18 of which are new contributors

Support for reading, writing, and converting Cosmology

Four new methods – read(), write(), from_format(), to_format() – have been added to the Cosmology class, enabling reading from and writing to files and converting between different Python objects. The methods use Astropy’s Unified I/O registry so custom formats can be registered. For further details see Read, Write, and Convert Cosmology Objects.

For writing the representation of a Cosmology to a file, the ECSV table format is available. Astropy is encouraging the use of this format as a standard interoperable way of storing and communicating cosmology parameters. For example, to write out cosmological parameters:

>>> from astropy.cosmology import Cosmology, Planck18
>>> Planck18.write('planck18.ecsv', format="ascii.ecsv")

and to read them back in:

>>> cosmo = Cosmology.read('planck18.ecsv', format="ascii.ecsv")
>>> cosmo == Planck18
True

The to_format() method can be used to convert cosmologies to other types such as dict, QTable, and Model. For example:

>>> ct = Planck18.to_format("astropy.table")
>>> ct
<QTable length=1>
  name        H0        Om0    Tcmb0    Neff    m_nu [3]    Ob0
         km / (Mpc s)            K                 eV
  str8     float64    float64 float64 float64   float64   float64
-------- ------------ ------- ------- ------- ----------- -------
Planck18        67.66 0.30966  2.7255   3.046 0.0 .. 0.06 0.04897

Models can be created from any redshift method of a Cosmology:

>>> model = Planck18.to_format("astropy.model", method="lookback_time")
>>> model
<FlatLambdaCDMCosmologyLookbackTimeModel(H0=67.66 km / (Mpc s), Om0=0.30966,
    Tcmb0=2.7255 K, Neff=3.046, m_nu=[0.  , 0.  , 0.06] eV, Ob0=0.04897,
    name='Planck18')>

For details, e.g. how to parse a QTable to a Cosmology, see Read, Write, and Convert Cosmology Objects.

Cosmology units module

A new module – astropy.cosmology.units – has been added to the cosmology subpackage for defining and collecting cosmological units and equivalencies. The unit littleh and equivalency with_H0 are deprecated from the main astropy.units subpackage and moved to cosmology.units. A new unit, redshift, is added for tracking factors of cosmological redshift. As this is a pseudo-unit an equivalency dimensionless_redshift is added (and enabled by default) to allow for redshift - dimensionless conversions. To convert between redshift and other cosmological distance measures, e.g. CMB temperature or comoving distance, the equivalency with_redshift is also added.

>>> import astropy.units as u
>>> import astropy.cosmology.units as cu
>>> z = 1100 * cu.redshift
>>> z.to(u.dimensionless_unscaled)
<Quantity 1100.>
>>> from astropy.cosmology import WMAP9
>>> equivalency = cu.with_redshift(WMAP9)  # construct equivalency
>>> z.to(u.K, equivalency)
<Quantity 3000.225 K>
>>> z.to(u.km / u.s / u.Mpc, equivalency)
<Quantity 1565637.40154275 km / (Mpc s)>
>>> z.to(cu.littleh, equivalency)
<Quantity 15656.37401543 littleh>
>>> z.to(u.Mpc, equivalency)
<Quantity 14004.03157418 Mpc>

with_redshift() is actually a composite of other equivalencies: redshift_distance(), redshift_hubble(), and redshift_temperature(), which may be used separately.

New Models

The following models have now been added:

A new module – astropy.modeling.spline – has been added to the modeling subpackage for defining spline models for astropy. Currently this only contains a one-dimensional spline model: Spline1D. Since splines have unique fitting requirements four fitters have been introduced:

import numpy as np
import matplotlib.pyplot as plt
from astropy.modeling.models import Spline1D
from astropy.modeling.fitting import (SplineInterpolateFitter,
                                      SplineSmoothingFitter,
                                      SplineExactKnotsFitter)

x = np.linspace(-3, 3, 50)
y = np.exp(-x**2) + 0.1 * np.random.randn(50)
xs = np.linspace(-3, 3, 1000)
t = [-1, 0, 1]
spl = Spline1D()

fitter = SplineInterpolateFitter()
spl1 = fitter(spl, x, y)

fitter = SplineSmoothingFitter()
spl2 = fitter(spl, x, y, s=0.5)

fitter = SplineExactKnotsFitter()
spl3 = fitter(spl, x, y, t=t)

plt.plot(x, y, 'ro', label="Data")
plt.plot(xs, spl1(xs), 'b-', label="Interpolating")
plt.plot(xs, spl2(xs), 'g-', label="Smoothing")
plt.plot(xs, spl3(xs), 'k-', label="Exact Knots [-1, 0, 1]")
plt.legend()
plt.show()

(png, svg, pdf)

../_images/5-0-1.png

Added support for dask arrays in tables

Dask arrays are now preserved instead of being converted to Numpy arrays when added to tables:

>>> from astropy.table import Table
>>> import dask.array as da
>>> t = Table()
>>> t['a'] = da.arange(1_000_000_000_000)
>>> t
<Table length=1000000000000>
     a
   int64
------------
           0
           1
           2
           3
           4
         ...
999999999995
999999999996
999999999997
999999999998
999999999999
>>> t['a'][100:200].compute()
array([100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,
       113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,
       126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138,
       139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151,
       152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164,
       165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177,
       178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190,
       191, 192, 193, 194, 195, 196, 197, 198, 199]...)

Added support for registering array-like objects as mixin columns

It is now possible to register functions (which we call mixin ‘handlers’) which can be used by astropy tables to convert, on-the-fly, any array-like object into a ‘mixin’ column that can be used in a table. As an example, this is used internally to provide the seamless integration of dask arrays into tables, as shown in Added support for dask arrays in tables.

For more information about how to write your own handlers and register them, see Registering array-like objects as mixin columns.

Support for reading and writing tables to Parquet format

The Unified File Read/Write Interface now supports reading and writing files in the Parquet format if the pyarrow package is installed. Apache Parquet is a columnar storage format related to the Hadoop ecosystem which supports a wide variety of data processing frameworks and programming languages. A key benefit of Parquet files is that each column is stored independently, and thus reading a subset of columns is fast and efficient. For more details see the Parquet section.

Support for reading and writing tables to MRT format

Astropy now supports reading and writing tables in the American Astronomical Society Journals’ Machine-Readable Table (MRT) format. This ASCII table format consists of single file with the table description header and the table data itself. MRT is similar to the CDS format standard, but differs in the table description sections and the lack of a separate ReadMe file.

The writing functionality applies special handling to the first SkyCoord column in the table to adhere to the MRT recommendations for presenting coordinate data. It also ensures that columns with units are written with the correct unit convention.

For details see the Machine-Readable Table Format section.

Support for masked quantity columns, including masked FITS columns with units

Masked quantities are now fully supported in tables. This includes QTable automatically converting MaskedColumn instances to MaskedQuantity, and Table doing the reverse.

A consequence of this is that when FITS tables are read into a QTable, any missing entries will be masked. Hence, when such columns have units, all NaN values will now be masked.

Converting SkyCoord to QTable

Astropy v5.0 adds the to_table() method to SkyCoord objects, which creates a QTable from the SkyCoord:

>>> from astropy import units as u
>>> from astropy.coordinates import SkyCoord
>>> from astropy.time import Time
>>> sc = SkyCoord(ra=[40, 70]*u.deg, dec=[0, -20]*u.deg,
...               obstime=Time([2000, 2010], format='jyear'))
>>> t =  sc.to_table()
>>> t
<QTable length=2>
   ra     dec   obstime
  deg     deg
float64 float64   Time
------- ------- -------
   40.0     0.0  2000.0
   70.0   -20.0  2010.0

Information about the coordinate frame is stored in the metadata of the table:

>>> t.meta
{'representation_type': 'spherical', 'frame': 'icrs'}

For more details see Converting a SkyCoord to a Table.

New Unified I/O architecture

The registry submodule has switched to a class-based architecture, allowing for the creation of custom registries. The three supported registry types are:

For backward compatibility all the methods on the read/write have corresponding module-level functions, which work with a default global read/write registry.

Full change log

To see a detailed list of all changes in version v5.0, including changes in API, please see the Full Changelog.

Contributors to the v5.0 release

The people who have contributed to the code for this release are:

  • Aarya Patil

  • Adam Ginsburg

  • Adrian Price-Whelan

  • Akeem *

  • Akshat Dixit *

  • Akshat1Nar *

  • Albert Y. Shih

  • Brett Morris

  • Brigitta Sipőcz

  • Christoph Gohlke

  • Conor MacBride *

  • Daniel Ryan

  • Daria Cara

  • David Stansby

  • Derek Homeier

  • Diego Asterio de Zaballa *

  • E. Madison Bray

  • E. Rykoff *

  • Ed Slavich

  • Eero Vaher

  • Emir *

  • Erik Tollerud

  • Gabriel Perren

  • Hans Moritz Günther

  • James Tocknell

  • Jason Segnini *

  • Jero Bado

  • Larry Bradley

  • Laurie Stephey *

  • Leo Singer

  • Markus Demleitner *

  • Marten van Kerkwijk

  • Matteo Bachetti

  • Maximilian Nöthe

  • Michael Brewer *

  • Mihai Cara

  • Nadia Dencheva

  • Nathaniel Starkman

  • Ole Streicher

  • Pey Lian Lim

  • Roy Smart *

  • Sam Van Kooten *

  • Simon Conseil

  • Stuart Mumford

  • Suyog Garg

  • Thomas Robitaille

  • Tom Aldcroft

  • Volodymyr Savchenko *

  • William Jamieson

  • Zé Vinicius

  • athul *

  • iamsoto *

  • luz paz *

  • odidev *

Where a * indicates that this release contains their first contribution to Astropy.