What’s New in Astropy 4.0?

Pre-Publication Planck 2018 Cosmological Parameters

A pre-publication version of the Planck 2018 cosmological parameters has been included based on the second version of the submitted paper. This will be replaced with a final version when the paper is accepted.

>>> from astropy.cosmology import Planck18_arXiv_v2
>>> Planck18_arXiv_v2.age(0)  
<Quantity 13.7868853 Gyr>

Improved Consistency of Physical Constants and Units

Physical constants and the units using them are based on measurements that improve over time and therefore are not “constant” numerically. Generally, you want to use the latest values, but sometimes it is necessary to reproduce earlier results by going back in time. For that purpose, we have now introduced new astropy.physical_constants and astropy.astronomical_constants science state objects, which can be used to enable previous versions of the constants, and to make sure that units and physical constants are self-consistent. For more details, see Collections of Constants (and Prior Versions).

Updates to Galactocentric Frame

Most coordinate frames implemented in astropy.coordinates have standard parameters that are set by IAU consensus (e.g., the ICRS frame). Unlike these, the Galactocentric coordinate frame does not have an absolute definition: its parameters (the solar motion and position relative to the Galactic center) are measurements that continue to be refined as newer stellar surveys are executed and analyzed. When it was added, the default parameter values used by the Galactocentric frame (i.e., the parameter values assumed when defining a frame without explicitly setting values, like galcen = Galactocentric()) were set to commonly used values at the time, but these are now somewhat out of date. With v4.0, we have added functionality for globally controlling the default parameter values used by this frame by setting the galactocentric_frame_defaults object with the name of a parameter set. The parameter set names can currently be one of "pre-v4.0" (to get the original, pre-version-4.0 values of the parameters), "v4.0" (to get a more modern set of values adopted in v4.0), and "latest" (which is currently an alias for "v4.0" and will always alias the most recent set of parameters).

If your code depends sensitively on the choice of Galactocentric frame parameters, make sure to explicitly set the parameter set in your code, for example, after importing astropy.coordinates:

>>> import astropy.coordinates as coord
>>> coord.galactocentric_frame_defaults.set('v4.0')  

The Galactocentric frame now also maintains a list of references to scientific papers for the default values of the frame attributes. For example, after adopting the v4.0 parameter set and defining a frame, we can retrieve the references (as a dictionary of links to ADS) for the parameters using the .frame_attribute_references attribute:

>>> import astropy.coordinates as coord
>>> coord.galactocentric_frame_defaults.set('v4.0')  
>>> galcen = coord.Galactocentric()
>>> galcen  
<Galactocentric Frame (galcen_coord=<ICRS Coordinate: (ra, dec) in deg
    (266.4051, -28.936175)>, galcen_distance=8.122 kpc, galcen_v_sun=(12.9, 245.6, 7.78) km / s, z_sun=20.8 pc, roll=0.0 deg)>
>>> galcen.frame_attribute_references
{'galcen_coord': 'http://adsabs.harvard.edu/abs/2004ApJ...616..872R',
 'galcen_distance': 'https://ui.adsabs.harvard.edu/abs/2018A%26A...615L..15G',
 'galcen_v_sun': ['https://ui.adsabs.harvard.edu/abs/2018RNAAS...2..210D',
 'z_sun': 'https://ui.adsabs.harvard.edu/abs/2019MNRAS.482.1417B'}

Note, however, if a frame parameter is set by the user, it is removed from the reference list:

>>> import astropy.units as u
>>> galcen = coord.Galactocentric(z_sun=10*u.pc)
>>> galcen.frame_attribute_references
{'galcen_coord': 'http://adsabs.harvard.edu/abs/2004ApJ...616..872R',
 'galcen_distance': 'https://ui.adsabs.harvard.edu/abs/2018A%26A...615L..15G',
 'galcen_v_sun': ['https://ui.adsabs.harvard.edu/abs/2018RNAAS...2..210D',

More information can be found in the documentation for the frame class: Galactocentric.

New ymdhms Time Format

A new Time format was added to allow convenient input and output of times via year, month, day, hour, minute, and second values. For example:

>>> from astropy.time import Time
>>> t = Time({'year': 2015, 'month': 2, 'day': 3,
...           'hour': 12, 'minute': 13, 'second': 14.567})
>>> t.iso
'2015-02-03 12:13:14.567'
>>> t.ymdhms.year

New Context Manager for Plotting Time Values

Matplotlib natively provides a mechanism for plotting dates and times on one or both of the axes, as described in Date tick labels. To make use of this, you can use the plot_date attribute of Time to get values in the time system used by Matplotlib.

However, in many cases, you will probably want to have more control over the precise scale and format to use for the tick labels, in which case you can make use of the time_support function which can be called either directly or as a context manager, and after which Time objects can be passed to matplotlib plotting functions. The axes are then automatically labeled with times formatted using the Time class:

import matplotlib.pyplot as plt
from astropy.time import Time
from astropy.visualization import time_support
time_support(format='isot', scale='tai')  # doctest: +IGNORE_OUTPUT
plt.figure(figsize=(5,3))  # doctest: +IGNORE_OUTPUT
plt.plot(Time([52000, 53000, 54000], format='mjd'), [1.2, 3.3, 2.3])  # doctest: +IGNORE_OUTPUT

(png, svg, pdf)


For more information, see Plotting times.

Support for Parsing High-Precision Values with Time

For numerical formats, Time can now be instantiated from strings, quadruple precision numpy floats (if available on a given platform), and Decimal instances. For instance, in the example below you can see how in a string we can give full precision, while entering the same number as a float gives precision loss:

>>> from astropy.time import Time
>>> t = Time('2450000.123456789012345', format='jd')
>>> t - Time(2450000, 0.123456789012345, format='jd')
<TimeDelta object: scale='tai' format='jd' value=0.0>
>>> t - Time(2450000.123456789012345, format='jd')  
<TimeDelta object: scale='tai' format='jd' value=-1.6829138083096495e-10>

You can also output values as string, etc.:

>>> t.to_value('jd', subfmt='str')

Improved Handling of Leap Second Updates

astropy now automatically checks for and applies new leap seconds the first time a Time is instantiated. This is done with the new LeapSeconds class, which can, in the hopefully unlikely case it is needed, also be used directly.

Major Improvements in Compatibility of Quantity Objects with NumPy Functions

While Quantity objects have worked well in arithmetic operations via numpy’s “universal functions” (ufuncs), for other numpy functions it has been a bit hit and miss. For instance, units would be lost when trying to concatenate quantities, make histograms, or in functions such as numpy.where.

For numpy version 1.17 and later, however, it is possible to override the behavior of numpy functions, and this is used in astropy 4.0 to make essentially all functions work as expected with quantities. If a numpy function does not work as expected, it is now a bug that we can fix!

Two concise examples:

>>> import numpy as np
>>> from astropy import units as u
>>> np.where([True, False, False], [1., 2., 3.]*u.m, 1.*u.cm)
<Quantity [1.  , 0.01, 0.01] m>
>>> np.hstack(([1., 2., 3.]*u.m, 1.*u.cm))
<Quantity [1.  , 2.  , 3.  , 0.01] m>


for NumPy 1.16, one can get the same behaviour by setting environment variable NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=1. For details, see NEP 18

Plotting 1D Profile Plots with WCSAxes

The astropy.visualization.wcsaxes module now supports plotting data with one-dimensional WCS (including 1D profiles extracted from higher dimensional objects). The following example shows a plot of a 1D profile extracted from a 3D spectral cube — because all world coordinates vary along that slice, all three coordinates are still shown in the final plot:

import matplotlib.pyplot as plt
import astropy.units as u
from astropy.wcs import WCS
from astropy.io import fits
from astropy.utils.data import get_pkg_data_filename

filename = get_pkg_data_filename('l1448/l1448_13co.fits')
hdu = fits.open(filename)[0]

ax = plt.subplot(projection=WCS(hdu.header), slices=(50, 'x', 'y'))
ax.imshow(hdu.data[:, :, 50])

(png, svg, pdf)


Default Labelling with WCSAxes

As seen in the example in Plotting 1D Profile Plots with WCSAxes, axis labels are now shown by default, using either the names of the coordinates axes, if available, or the physical types of the axes. To disable this, you can use:


Or you can also set the axis label to what you want instead:

ax.coords[0].set_axislabel("Right Ascension")
ax.coords[2].set_axislabel("Velocity (m/s)")

New Function to Fit WCS to Pairs of Pixel/World Coordinates

A new function astropy.wcs.utils.fit_wcs_from_points() has been added to fit a (FITS) WCS to a set of points for which both pixel and world coordinates are available. For instance, if we have an image in which four stars have known pixel and celestial coordinates:

>>> import numpy as np
>>> from astropy.coordinates import SkyCoord
>>> stars_pixel = [np.array([153, 64, 593, 663]),
...                np.array([581, 199, 190, 445])]
>>> stars_world = SkyCoord([266.729, 266.872, 266.031, 265.921],
...                        [-28.627, -29.156, -29.170, -28.815],
...                        unit='deg', frame='fk5')

we can find the best-fitting WCS with:

.. doctest-requires:: scipy
>>> from astropy.wcs.utils import fit_wcs_from_points
>>> fit_wcs_from_points(stars_pixel, stars_world)  # doctest: +FLOAT_CMP
WCS Keywords
Number of WCS axes: 2
CRVAL : 266.3952562116589  -28.89933479456074
CRPIX : 364.8753661483385  385.9573650918672
CD1_1 CD1_2  : -0.001388743579175224  4.580696254922365e-08
CD2_1 CD2_2  : -8.098819668907673e-07  0.0013876745578212755
NAXIS : 599  391

Support for WCS Transformations between Pixel and Time Values

The WCS.world_to_pixel and WCS.pixel_to_world methods can now take and return Time objects for WCS transformations that involve time:

>>> from astropy.io import fits
>>> from astropy.wcs import WCS
>>> header = fits.Header()
>>> header['CTYPE1'] = 'TIME'
>>> header['CDELT1'] = 86400.
>>> header['MJDREF'] = 58788.
>>> wcs = WCS(header)
>>> wcs.pixel_to_world([2, 3, 4])
<Time object: scale='utc' format='mjd' value=[58791. 58792. 58793.]>
>>> wcs.world_to_pixel(Time('2019-11-02T10:30:22'))

Improvements to Folding for Time Series

The TimeSeries.fold method now includes more options for controlling the resulting phase values. First, the midpoint_epoch argument has been renamed to epoch_time so as to be more general, and the epoch_phase can be used to specify the phase at which the epoch is given. In addition, a new wrap_phase argument can be used to specify at what phase to wrap — for example, if this is set to half the period, the resulting phase will go from minus half the period to half the period, whereas if it is set to the period the resulting phase will go from zero to the period:

>>> from astropy import units as u
>>> from astropy.timeseries import TimeSeries
>>> ts = TimeSeries(time_start='2019-11-01T00:00:00', time_delta=0.3 * u.day,
...                 n_samples=10)
>>> tf1 = ts.fold(1 * u.day, epoch_time='2019-11-01T12:00:00',
...               wrap_phase=1 * u.day)
>>> tf1  
<TimeSeries length=10>

Finally, the new normalize_phase keyword argument can be used to specify whether the final phase should be a relative time or whether it should be normalized to a dimensionless value in the range 0 to 1:

>>> tf2 = ts.fold(1 * u.day, epoch_time='2019-11-01T12:00:00',
...               normalize_phase=True)
>>> tf2  
<TimeSeries length=10>


New Table Methods and Options

A new method dstack was added to allow depth-wise stacking of tables to turn a list of similar tables into a single “3D” table with shape (rows, columns, depth).

A new method to compare tables values_equal was added to allow element-wise comparison of a table to either another table, a list of values, or a single value. This returns a new Table with the boolean result of the comparisons.

The join operation now supports Cartesian joins, enumerating all possible combinations of the left and right table rows.

A Table can now be initialized with a list of dict where the dict keys are not the same in every row. The table column names are the set of all keys found in the input data, and any missing key/value pairs are turned into missing data in the table.

The add_column and add_columns methods can now accept any object(s) which can be converted or broadcasted to a valid column for the table. Previously, these methods required a valid Column or mixin column object.

Improvements to Performance for Tables

A number of performance improvements were introduced in version 4.0 that can substantially improve the speed of Table manipulations.

A key area was the handling of replacing and adding columns, which is now two to ten times faster in common cases. The implementation was changed so that the time for replacing or adding is independent of the number of existing columns (like a dict). This means you can now efficiently build a table from scratch by creating an empty table and then adding columns one at a time.

Another improvement was in the performance of table and column slicing. In addition to internal implementation changes, there was a change to reduce unnecessary copy and deepcopy of table and column meta attributes. In particular, table or column slices will now get a shallow key-only copy of the metadata instead of a deep copy.

Table row access speed was improved by a factor of a few, and getting the length of a table is now typically three to ten times faster. A new method iterrows was added to make row-wise iteration even faster for the common case of only needing a subset of the available columns:

>>> from astropy.table.table_helpers import simple_table
>>> t = simple_table(size=2, cols=10)
>>> print(t)
a   b   c   d   e   f   g   h   i   j
--- --- --- --- --- --- --- --- --- ---
    1 1.0   c   4 4.0   f   7 7.0   i  10
    2 2.0   d   5 5.0   g   8 8.0   j  11
>>> for a, f in t.iterrows('a', 'f'):
...     print(a, f)
1 f
2 g

New Models

The following models have now been added:

Downloading and Caching Files from the Internet

The existing download_file() mechanism has been substantially upgraded to be more reliable and more capable, and tools have been added to manage the collection of cached downloaded files. Most notably:

  • download_file() can accept a list of locations where the file can be obtained; wherever it was obtained it will be indexed under its “official” location.
  • download_file() can be told, using cache="update", to check the Internet to see whether a new version of the file is available, and if so, update the version in the cache; if something goes wrong the cache is left intact.
  • Concurrent use of the cache by multiple processes will be more reliable, and download_files_in_parallel() will allow you to use this within one process.
  • Part or all of the cache can be exported to or imported from a ZIP file with export_download_cache() and import_download_cache(). This can be useful for setting up machines that will not have an Internet connection.
  • If the cache is behaving oddly check_download_cache() can be used to try to diagnose any problems, and clear_download_cache() can remove problem objects or completely get rid of a damaged cache.
  • The amount of space being used by the cache is available with cache_total_size().

This mechanism is also available to other packages that want to use it, so most functions now also accept a pkgname argument, which allows different packages to maintain separate caches.

API Changes in astropy.modeling

A number of significant changes have been made to modeling API as a result of reworking how parameters and compound models work.

  • It is no longer possible to create compound model classes (as opposed to compound model instances).
  • Parameters now hold their values directly with the consequence that compound models share the same parameter instances as the constituent model they are constructed from (previously the values were copied and changes to one or the other had no effect on the corresponding model).
  • In compound models, the constituent models are references, not copies (if copies are desired, an explicit copy() should be used in the compound model expression).

There are other more minor changes to the API that are detailed in Changes to Modeling in v4.0:

  • inputs and outputs were deprecated as class variables and are instance variables, while n_inputs and n_outputs are now class variables. As a result inputs and outputs of a model can be renamed.
  • Assigning slices of the model parameter array does not automatically get reflected in parameter values
  • Previously it was possible to use arbitrary slices on compound models (which had the possibility of returning submodes with entirely different meanings than they had in the original compound model). Now only a restricted set of slices is permitted.
  • Use of “inputed” units is much more restricted. Previously these could end up with unexpected units being assigned.
  • Many private methods have been added, changed, or deleted.

In addition, a new BlackBody class has been added and replaces the now-deprecated BlackBody1D class and the blackbody_nu() and blackbody_lambda() functions. To find out more about the new class, see BlackBody, and for information about the correspondance between BlackBody and the deprecated class and functions, see Blackbody Module (deprecated capabilities).

API Changes in astropy.table

The handling of masked columns in tables has changed in a way that may impact program behavior. Now a Table object with masked=False may contain both Column and MaskedColumn objects, and adding a masked column or row to a table no longer “upgrades” the table and all other columns to masked. This means that tables with masked data which are read via Table.read() will now always have masked=False, though specific columns will be masked as needed. The same applies to the output of table operations like join, vstack, and hstack. Two new table properties has_masked_columns and has_masked_values were added. See the Masking change in astropy 4.0 section for details.

As noted earlier, the handling of table and column meta attributes has changed and users no longer get deep copies in most cases.

API Changes in astropy.uncertainty

For the experimental Distribution class, the earlier pdf_mean, pdf_var, etc., properties were turned into methods, both for consistency with the numpy methods that are used underneath, and to allow users to pass on parameters (such as the degrees of freedom ddof for pdf_var()).

While the module remains experimental, and further enhancements and refactoring are planned, we do not foresee any further significant changes in the API.

Full change log

To see a detailed list of all changes in version v4.0, including changes in API, please see the Full Changelog.

Contributors to the v4.0 release

  • Aarya Patil
  • Adam Ginsburg
  • Adrian Price-Whelan
  • Albert Y. Shih
  • Alex Conley
  • Anne Archibald
  • Arthur Eigenbrot *
  • Benjamin Alan Weaver
  • Brett M. Morris *
  • Brigitta Sipőcz
  • Bryce Kalmbach *
  • Christoph Deil
  • Clara Brasseur
  • Clare Shanahan *
  • Dan Foreman-Mackey
  • Daria Cara
  • David Shupe
  • David Stansby
  • Derek Homeier
  • Douglas Burke
  • Drew Leonard *
  • Erik M. Bray
  • Erik Tollerud
  • Frédéric Chapoton
  • Geert Barentsen
  • Gregory Dubois-Felsmann *
  • Hannes Breytenbach
  • Hans Moritz Günther
  • Harry Ferguson
  • Himanshu Pathak
  • James Davies
  • John Fisher *
  • John Parejko
  • Johnny Greco
  • Juan Luis Cano Rodríguez
  • Julien Woillez
  • Karl Gordon
  • Kewei Li *
  • Larry Bradley
  • Lauren Glattly
  • Leo Singer
  • Lia Corrales *
  • M Atakan Gürkan *
  • Mark Fardal *
  • Marten van Kerkwijk
  • Matteo Bachetti
  • Matthew Craig
  • Maximilian Nöthe
  • Michael Seifert
  • Mihai Cara
  • Nadia Dencheva
  • Nora Luetzgendorf *
  • Perry Greenfield
  • Pey Lian Lim
  • Rasmus Handberg *
  • Rui Xue *
  • SF Graves *
  • Sadie Bartholomew *
  • Semyeong Oh
  • Shreyas Bapat *
  • Simon Conseil
  • Simon Torres *
  • Stuart Mumford
  • Thomas Robitaille
  • Tiffany Jansen *
  • Tom Aldcroft
  • Tom Donaldson *
  • Tom J Wilson *
  • Vishnunarayan K I
  • Wilfred Tyler Gee *
  • Yash Sharma *
  • Yingqi Ying *
  • Zachary Kurtz *
  • rtolesnikov *