See also
Array creation routines
Introduction#
There are 6 general mechanisms for creating arrays:
Conversion from other Python structures (i.e. lists and tuples)
Intrinsic NumPy array creation functions (e.g. arange, ones, zeros,etc.)
Replicating, joining, or mutating existing arrays
Reading arrays from disk, either from standard or custom formats
Creating arrays from raw bytes through the use of strings or buffers
Use of special library functions (e.g., random)
You can use these methods to create ndarrays or Structured arrays.This document will cover general methods for ndarray creation.
1) Converting Python sequences to NumPy arrays#
NumPy arrays can be defined using Python sequences such as lists andtuples. Lists and tuples are defined using [...] and (...),respectively. Lists and tuples can define ndarray creation:
a list of numbers will create a 1D array,
a list of lists will create a 2D array,
further nested lists will create higher-dimensional arrays. In general, any array object is called an ndarray in NumPy.
>>> import numpy as np>>> a1D = np.array([1, 2, 3, 4])>>> a2D = np.array([[1, 2], [3, 4]])>>> a3D = np.array([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
When you use numpy.array to define a new array, you shouldconsider the dtype of the elements in the array,which can be specified explicitly. This feature gives youmore control over the underlying data structures and how the elementsare handled in C/C++ functions.When values do not fit and you are using a dtype, NumPy may raise anerror:
>>> import numpy as np>>> np.array([127, 128, 129], dtype=np.int8)Traceback (most recent call last):...OverflowError: Python integer 128 out of bounds for int8
An 8-bit signed integer represents integers from -128 to 127.Assigning the int8 array to integers outside of this range resultsin overflow. This feature can often be misunderstood. If youperform calculations with mismatching dtypes, you can get unwantedresults, for example:
>>> import numpy as np>>> a = np.array([2, 3, 4], dtype=np.uint32)>>> b = np.array([5, 6, 7], dtype=np.uint32)>>> c_unsigned32 = a - b>>> print('unsigned c:', c_unsigned32, c_unsigned32.dtype)unsigned c: [4294967293 4294967293 4294967293] uint32>>> c_signed32 = a - b.astype(np.int32)>>> print('signed c:', c_signed32, c_signed32.dtype)signed c: [-3 -3 -3] int64
Notice when you perform operations with two arrays of the samedtype: uint32, the resulting array is the same type. When youperform operations with different dtype, NumPy willassign a new type that satisfies all of the array elements involved inthe computation, here uint32 and int32 can both be represented inas int64.
The default NumPy behavior is to create arrays in either 32 or 64-bit signedintegers (platform dependent and matches C long size) or double precisionfloating point numbers. If you expect yourinteger arrays to be a specific type, then you need to specify the dtype whileyou create the array.
2) Intrinsic NumPy array creation functions#
NumPy has over 40 built-in functions for creating arrays as laidout in the Array creation routines.These functions can be split into roughly three categories, based on thedimension of the array they create:
1D arrays
2D arrays
ndarrays
1 - 1D array creation functions#
The 1D array creation functions e.g. numpy.linspace andnumpy.arange generally need at least two inputs, start andstop.
numpy.arange creates arrays with regularly incrementing values.Check the documentation for complete information and examples. A fewexamples are shown:
>>> import numpy as np>>> np.arange(10)array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])>>> np.arange(2, 10, dtype=float)array([2., 3., 4., 5., 6., 7., 8., 9.])>>> np.arange(2, 3, 0.1)array([2. , 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9])
Note: best practice for numpy.arange is to use integer start, end, andstep values. There are some subtleties regarding dtype. In the secondexample, the dtype is defined. In the third example, the array isdtype=float to accommodate the step size of 0.1. Due to roundoff error,the stop value is sometimes included.
numpy.linspace will create arrays with a specified number of elements, andspaced equally between the specified beginning and end values. Forexample:
>>> import numpy as np>>> np.linspace(1., 4., 6)array([1. , 1.6, 2.2, 2.8, 3.4, 4. ])
The advantage of this creation function is that you guarantee thenumber of elements and the starting and end point. The previousarange(start, stop, step) will not include the value stop.
2 - 2D array creation functions#
The 2D array creation functions e.g. numpy.eye, numpy.diag, and numpy.vanderdefine properties of special matrices represented as 2D arrays.
np.eye(n, m) defines a 2D identity matrix. The elements where i=j (row index and column index are equal) are 1and the rest are 0, as such:
>>> import numpy as np>>> np.eye(3)array([[1., 0., 0.], [0., 1., 0.], [0., 0., 1.]])>>> np.eye(3, 5)array([[1., 0., 0., 0., 0.], [0., 1., 0., 0., 0.], [0., 0., 1., 0., 0.]])
numpy.diag can define either a square 2D array with given values alongthe diagonal or if given a 2D array returns a 1D array that isonly the diagonal elements. The two array creation functions can be helpful whiledoing linear algebra, as such:
>>> import numpy as np>>> np.diag([1, 2, 3])array([[1, 0, 0], [0, 2, 0], [0, 0, 3]])>>> np.diag([1, 2, 3], 1)array([[0, 1, 0, 0], [0, 0, 2, 0], [0, 0, 0, 3], [0, 0, 0, 0]])>>> a = np.array([[1, 2], [3, 4]])>>> np.diag(a)array([1, 4])
vander(x, n) defines a Vandermonde matrix as a 2D NumPy array. Each columnof the Vandermonde matrix is a decreasing power of the input 1D array orlist or tuple,x where the highest polynomial order is n-1. This array creationroutine is helpful in generating linear least squares models, as such:
>>> import numpy as np>>> np.vander(np.linspace(0, 2, 5), 2)array([[0. , 1. ], [0.5, 1. ], [1. , 1. ], [1.5, 1. ], [2. , 1. ]])>>> np.vander([1, 2, 3, 4], 2)array([[1, 1], [2, 1], [3, 1], [4, 1]])>>> np.vander((1, 2, 3, 4), 4)array([[ 1, 1, 1, 1], [ 8, 4, 2, 1], [27, 9, 3, 1], [64, 16, 4, 1]])
3 - general ndarray creation functions#
The ndarray creation functions e.g. numpy.ones,numpy.zeros, and random definearrays based upon the desired shape. The ndarray creation functionscan create arrays with any dimension by specifying how many dimensionsand length along that dimension in a tuple or list.
numpy.zeros will create an array filled with 0 values with thespecified shape. The default dtype is float64:
>>> import numpy as np>>> np.zeros((2, 3))array([[0., 0., 0.], [0., 0., 0.]])>>> np.zeros((2, 3, 2))array([[[0., 0.], [0., 0.], [0., 0.]], [[0., 0.], [0., 0.], [0., 0.]]])
numpy.ones will create an array filled with 1 values. It is identical tozeros in all other respects as such:
>>> import numpy as np>>> np.ones((2, 3))array([[1., 1., 1.], [1., 1., 1.]])>>> np.ones((2, 3, 2))array([[[1., 1.], [1., 1.], [1., 1.]], [[1., 1.], [1., 1.], [1., 1.]]])
The random method of the result ofdefault_rng will create an array filled with randomvalues between 0 and 1. It is included with the numpy.randomlibrary. Below, two arrays are created with shapes (2,3) and (2,3,2),respectively. The seed is set to 42 so you can reproduce thesepseudorandom numbers:
>>> import numpy as np>>> from numpy.random import default_rng>>> default_rng(42).random((2,3))array([[0.77395605, 0.43887844, 0.85859792], [0.69736803, 0.09417735, 0.97562235]])>>> default_rng(42).random((2,3,2))array([[[0.77395605, 0.43887844], [0.85859792, 0.69736803], [0.09417735, 0.97562235]], [[0.7611397 , 0.78606431], [0.12811363, 0.45038594], [0.37079802, 0.92676499]]])
numpy.indices will create a set of arrays (stacked as a one-higherdimensioned array), one per dimension with each representing variation in thatdimension:
>>> import numpy as np>>> np.indices((3,3))array([[[0, 0, 0], [1, 1, 1], [2, 2, 2]], [[0, 1, 2], [0, 1, 2], [0, 1, 2]]])
This is particularly useful for evaluating functions of multiple dimensions ona regular grid.
3) Replicating, joining, or mutating existing arrays#
Once you have created arrays, you can replicate, join, or mutate thoseexisting arrays to create new arrays. When you assign an array or itselements to a new variable, you have to explicitly numpy.copy the array,otherwise the variable is a view into the original array. Consider thefollowing example:
>>> import numpy as np>>> a = np.array([1, 2, 3, 4, 5, 6])>>> b = a[:2]>>> b += 1>>> print('a =', a, '; b =', b)a = [2 3 3 4 5 6] ; b = [2 3]
In this example, you did not create a new array. You created a variable,b that viewed the first 2 elements of a. When you added 1 to b youwould get the same result by adding 1 to a[:2]. If you want to create anew array, use the numpy.copy array creation routine as such:
>>> import numpy as np>>> a = np.array([1, 2, 3, 4])>>> b = a[:2].copy()>>> b += 1>>> print('a = ', a, 'b = ', b)a = [1 2 3 4] b = [2 3]
For more information and examples look at Copies and Views.
There are a number of routines to join existing arrays e.g. numpy.vstack,numpy.hstack, and numpy.block. Here is an example of joining four 2-by-2arrays into a 4-by-4 array using block:
>>> import numpy as np>>> A = np.ones((2, 2))>>> B = np.eye(2, 2)>>> C = np.zeros((2, 2))>>> D = np.diag((-3, -4))>>> np.block([[A, B], [C, D]])array([[ 1., 1., 1., 0.], [ 1., 1., 0., 1.], [ 0., 0., -3., 0.], [ 0., 0., 0., -4.]])
Other routines use similar syntax to join ndarrays. Check theroutine’s documentation for further examples and syntax.
4) Reading arrays from disk, either from standard or custom formats#
This is the most common case of large array creation. The details dependgreatly on the format of data on disk. This section gives general pointers onhow to handle various formats. For more detailed examples of IO look atHow to Read and Write files.
Standard binary formats#
Various fields have standard formats for array data. The following lists theones with known Python libraries to read them and return NumPy arrays (theremay be others for which it is possible to read and convert to NumPy arrays socheck the last section as well)
HDF5: h5pyFITS: Astropy
Examples of formats that cannot be read directly but for which it is not hard toconvert are those formats supported by libraries like PIL (able to read andwrite many image formats such as jpg, png, etc).
Common ASCII formats#
Delimited files such as comma separated value (csv) and tab separatedvalue (tsv) files are used for programs like Excel and LabView. Pythonfunctions can read and parse these files line-by-line. NumPy has twostandard routines for importing a file with delimited data numpy.loadtxtand numpy.genfromtxt. These functions have more involved use cases inReading and writing files. A simple example given a simple.csv:
$ cat simple.csvx, y0, 01, 12, 43, 9
Importing simple.csv is accomplished using numpy.loadtxt:
>>> import numpy as np>>> np.loadtxt('simple.csv', delimiter = ',', skiprows = 1) array([[0., 0.], [1., 1.], [2., 4.], [3., 9.]])
More generic ASCII files can be read using scipy.io and Pandas.
5) Creating arrays from raw bytes through the use of strings or buffers#
There are a variety of approaches one can use. If the file has a relativelysimple format then one can write a simple I/O library and use the NumPyfromfile() function and .tofile() method to read and write NumPy arraysdirectly (mind your byteorder though!) If a good C or C++ library exists thatread the data, one can wrap that library with a variety of techniques thoughthat certainly is much more work and requires significantly more advancedknowledge to interface with C or C++.
6) Use of special library functions (e.g., SciPy, pandas, and OpenCV)#
NumPy is the fundamental library for array containers in the Python Scientific Computingstack. Many Python libraries, including SciPy, Pandas, and OpenCV, use NumPy ndarraysas the common format for data exchange, These libraries can create,operate on, and work with NumPy arrays.
