Clash Royale CLAN TAG#URR8PPP

In Python, how do I read a file line-by-line into a list?

How do I read every line of a file in Python and store each line as an element in a list?

I want to read the file line by line and append each line to the end of the list.

34 Answers
34

with open(fname) as f:
content = f.readlines()
# you may also want to remove whitespace characters like `n` at the end of each line
content = [x.strip() for x in content]


file.readlines()

for

lines = [line.rstrip('n') for line in file]

readlines()

for line in f:

line

.rstrip()

See Input and Ouput:

with open('filename') as f:
lines = f.readlines()


or with stripping the newline character:

lines = [line.rstrip('n') for line in open('filename')]


^{Editor's note: This answer's original whitespace-stripping command, line.strip(), as implied by Janus Troelsen's comment, would remove all leading and trailing whitespace, not just the trailing n.}

line.strip()

n

lines = (line.rstrip('n') for line in open(filename))

lines = [line.rstrip('n') for line in open(filename)]

with open('filename') as f: lines = [line.rstrip('n') for line in f]

This is more explicit than necessary, but does what you want.

with open("file.txt", "r") as ins:
array =
for line in ins:
array.append(line)


This will yield an "array" of lines from the file.

lines = tuple(open(filename, 'r'))


open

tuple

lines

lines = open(filename).read().split('n')

lines = open(filename).read().splitlines()

list

tuple

from sys import getsizeof as g; i = [None] * 1000; round((g(list(i)) / g(tuple(i)) - 1) * 100, 2)

tuple

list

from timeit import timeit as t; round((t('tuple(i)', 'i = [None] * 1000') / t('list(i)', 'i = [None] * 1000') - 1) * 100, 2)

If you want the n included:

n

with open(fname) as f:
content = f.readlines()


If you do not want n included:

n

with open(fname) as f:
content = f.read().splitlines()


You could simply do the following, as has been suggested:

with open('/your/path/file') as f:
my_lines = f.readlines()


Note that this approach has 2 downsides:

1) You store all the lines in memory. In the general case, this is a very bad idea. The file could be very large, and you could run out of memory. Even if it's not large, it is simply a waste of memory.

2) This does not allow processing of each line as you read them. So if you process your lines after this, it is not efficient (requires two passes rather than one).

A better approach for the general case would be the following:

with open('/your/path/file') as f:
for line in f:
process(line)


Where you define your process function any way you want. For example:

def process(line):
if 'save the world' in line.lower():
superman.save_the_world()


(The implementation of the Superman class is left as an exercise for you).

Superman

This will work nicely for any file size and you go through your file in just 1 pass. This is typically how generic parsers will work.

open('file_path', 'r+')

If you don't care about closing the file, this one-liner works:

lines = open('file.txt').read().split("n")


The traditional way:

fp = open('file.txt') # Open file on read mode
lines = fp.read().split("n") # Create a list containing all lines
fp.close() # Close file


Using with (recommended):

with

with open('file.txt') as fp:
lines = fp.read().split("n")


with

fp.close()

This should encapsulate the open command.

array =
with open("file.txt", "r") as f:
for line in f:
array.append(line)


Data into list

Assume that we have a text file with our data like in the following lines:

line 1
line 2
line 3


python

>>> with open("myfile.txt", encoding="utf-8") as file:
... x = [l.strip() for l in file]
>>> x
['line 1','line 2','line 3']


x =
with open("myfile.txt") as file:
for l in file:
x.append(l.strip())


>>> x = open("myfile.txt").read().splitlines()
>>> x
['line 1', 'line 2', 'line 3']


>>> y = [x.rstrip() for x in open("my_file.txt")]
>>> y
['line 1','line 2','line 3']


encoding="utf-8"

Clean and Pythonic Way of Reading the Lines of a File Into a List

First and foremost, you should focus on opening your file and reading its contents in an efficient and pythonic way. Here is an example of the way I personally DO NOT prefer:

infile = open('my_file.txt', 'r') # Open the file for reading.

data = infile.read() # Read the contents of the file.

infile.close() # Close the file since we're done using it.


Instead, I prefer the below method of opening files for both reading and writing as it
is very clean, and does not require an extra step of closing the file
once you are done using it. In the statement below, we're opening the file
for reading, and assigning it to the variable 'infile.' Once the code within
this statement has finished running, the file will be automatically closed.

# Open the file for reading.
with open('my_file.txt', 'r') as infile:

data = infile.read() # Read the contents of the file into memory.


Now we need to focus on bringing this data into a Python List because they are iterable, efficient, and flexible. In your case, the desired goal is to bring each line of the text file into a separate element. To accomplish this, we will use the splitlines() method as follows:

# Return a list of the lines, breaking at line boundaries.
my_list = data.splitlines()


The Final Product:

# Open the file for reading.
with open('my_file.txt', 'r') as infile:

data = infile.read() # Read the contents of the file into memory.

# Return a list of the lines, breaking at line boundaries.
my_list = data.splitlines()


Testing Our Code:

A fost odatã ca-n povesti,
A fost ca niciodatã,
Din rude mãri împãrãtesti,
O prea frumoasã fatã.


print my_list # Print the list.

# Print each line in the list.
for line in my_list:
print line

# Print the fourth element in this list.
print my_list[3]


['A fost odatxc3xa3 ca-n povesti,', 'A fost ca niciodatxc3xa3,',
'Din rude mxc3xa3ri xc3xaempxc3xa3rxc3xa3testi,', 'O prea
frumoasxc3xa3 fatxc3xa3.']

A fost odatã ca-n povesti, A fost ca niciodatã, Din rude mãri
împãrãtesti, O prea frumoasã fatã.

O prea frumoasã fatã.


I'd do it like this.

lines =
with open("myfile.txt") as f:
for line in f:
lines.append(line)


To read a file into a list you need to do three things:

Fortunately Python makes it very easy to do these things so the shortest way to read a file into a list is:

lst = list(open(filename))


However I'll add some more explanation.

I assume that you want to open a specific file and you don't deal directly with a file-handle (or a file-like-handle). The most commonly used function to open a file in Python is open, it takes one mandatory argument and two optional ones in Python 2.7:

open

The filename should be a string that represents the path to the file. For example:

open('afile') # opens the file named afile in the current working directory
open('adir/afile') # relative path (relative to the current working directory)
open('C:/users/aname/afile') # absolute path (windows)
open('/usr/local/afile') # absolute path (linux)


Note that the file extension needs to be specified. This is especially important for Windows users because file extensions like .txt or .doc, etc. are hidden by default when viewed in the explorer.

.txt

.doc

The second argument is the mode, it's r by default which means "read-only". That's exactly what you need in your case.

mode

r

But in case you actually want to create a file and/or write to a file you'll need a different argument here. There is an excellent answer if you want an overview.

For reading a file you can omit the mode or pass it in explicitly:

mode

open(filename)
open(filename, 'r')


Both will open the file in read-only mode. In case you want to read in a binary file on Windows you need to use the mode rb:

rb

open(filename, 'rb')


On other platforms the 'b' (binary mode) is simply ignored.

'b'

Now that I've shown how to open the file, let's talk about the fact that you always need to close it again. Otherwise it will keep an open file-handle to the file until the process exits (or Python garbages the file-handle).

open

close

While you could use:

f = open(filename)
# ... do stuff with f
f.close()


That will fail to close the file when something between open and close throws an exception. You could avoid that by using a try and finally:

open

close

try

finally

f = open(filename)
# nothing in between!
try:
# do stuff with f
finally:
f.close()


However Python provides context managers that have a prettier syntax (but for open it's almost identical to the try and finally above):

open

try

finally

with open(filename) as f:
# do stuff with f
# The file is always closed after the with-scope ends.


The last approach is the recommended approach to open a file in Python!

Okay, you've opened the file, now how to read it?

The open function returns a file object and it supports Pythons iteration protocol. Each iteration will give you a line:

open

file

with open(filename) as f:
for line in f:
print(line)


This will print each line of the file. Note however that each line will contain a newline character n at the end (you might want to check if your Python is built with universal newlines support - otherwise you could also have rn on Windows or r on Mac as newlines). If you don't want that you can could simply remove the last character (or the last two characters on Windows):

n

rn

r

with open(filename) as f:
for line in f:
print(line[:-1])


But the last line doesn't necessarily has a trailing newline, so one shouldn't use that. One could check if it ends with a trailing newline and if so remove it:

with open(filename) as f:
for line in f:
if line.endswith('n'):
line = line[:-1]
print(line)


But you could simply remove all whitespaces (including the n character) from the end of the string, this will also remove all other trailing whitespaces so you have to be careful if these are important:

n

with open(filename) as f:
for line in f:
print(f.rstrip())


However if the lines end with rn (Windows "newlines") that .rstrip() will also take care of the r!

rn

.rstrip()

r

Now that you know how to open the file and read it, it's time to store the contents in a list. The simplest option would be to use the list function:

list

with open(filename) as f:
lst = list(f)


In case you want to strip the trailing newlines you could use a list comprehension instead:

with open(filename) as f:
lst = [line.rstrip() for line in f]


Or even simpler: The .readlines() method of the file object by default returns a list of the lines:

.readlines()

file

list

with open(filename) as f:
lst = f.readlines()


This will also include the trailing newline characters, if you don't want them I would recommend the [line.rstrip() for line in f] approach because it avoids keeping two lists containing all the lines in memory.

[line.rstrip() for line in f]

There's an additional option to get the desired output, however it's rather "suboptimal": read the complete file in a string and then split on newlines:

read

with open(filename) as f:
lst = f.read().split('n')


or:

with open(filename) as f:
lst = f.read().splitlines()


These take care of the trailing newlines automatically because the split character isn't included. However they are not ideal because you keep the file as string and as a list of lines in memory!

split

with open(...) as f

file

for line in the_file_object:

readlines()

Here's one more option by using list comprehensions on files;

lines = [line.rstrip() for line in open('file.txt')]


This should be more efficient way as the most of the work is done inside the Python interpreter.

rstrip()

n

.rstrip('n')

Another option is numpy.genfromtxt, for example:

numpy.genfromtxt

import numpy as np
data = np.genfromtxt("yourfile.dat",delimiter="n")


This will make data a NumPy array with as many rows as are in your file.

data

If you'd like to read a file from the command line or from stdin, you can also use the fileinput module:

fileinput

# reader.py
import fileinput

content =
for line in fileinput.input():
content.append(line.strip())

fileinput.close()


Pass files to it like so:

$ python reader.py textfile.txt


Read more here: http://docs.python.org/2/library/fileinput.html

The simplest way to do it

A simple way is to:

In one line, that would give:

lines = open('C:/path/file.txt').read().splitlines()


f = open("your_file.txt",'r')
out = f.readlines() # will append in the list out


Now variable out is a list (array) of what you want. You could either do:

for line in out:
print line


or

for line in f:
print line


you'll get the same results.

Just use the splitlines() functions. Here is an example.

inp = "file.txt"
data = open(inp)
dat = data.read()
lst = dat.splitlines()
print lst
# print(lst) # for python 3


In the output you will have the list of lines.

A real easy way:

with open(file) as g:
stuff = g.readlines()


If you want to make it a fully-fledged program, type this in:

file = raw_input ("Enter EXACT file name: ")
with open(file) as g:
stuff = g.readlines()
print (stuff)
exit = raw_input("Press enter when you are done.")


For some reason, it doesn't read .py files properly.

Read and write text files with Python 2 and Python 3; it works with Unicode

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

# Define data
lines = [' A first string ',
'A Unicode sample: €',
'German: äöüß']

# Write text file
with open('file.txt', 'w') as fp:
fp.write('n'.join(lines))

# Read text file
with open('file.txt', 'r') as fp:
read_lines = fp.readlines()
read_lines = [line.rstrip('n') for line in read_lines]

print(lines == read_lines)


Things to notice:

with

.strip()

.rstrip()

lines

.txt

.txt

For your application, the following might be important:

See also: Comparison of data serialization formats

In case you are rather looking for a way to make configuration files, you might want to read my short article Configuration files in Python.

You can just open your file for reading using:

file1 = open("filename","r")
# And for reading use
lines = file1.readlines()
file1.close()


The list lines will contain all your lines as individual elements, and you can call a specific element using lines["linenumber-1"] as Python starts its counting from 0.

lines

lines["linenumber-1"]

If you want to are faced with a very large / huge file and want to read faster (imagine you are in a Topcoder/Hackerrank coding competition), you might read a considerably bigger chunk of lines into a memory buffer at one time, rather than just iterate line by line at file level.

buffersize = 2**16
with open(path) as f:
while True:
lines_buffer = f.readlines(buffersize)
if not lines_buffer:
break
for line in lines_buffer:
process(line)


process(line)

print(line)

To my knowledge Python doesn't have a native array data structure. But it does support the list data structure which is much simpler to use than an array.

array = #declaring a list with name '**array**'
with open(PATH,'r') as reader :
for line in reader :
array.append(line)


array

Use this:

import pandas as pd
data = pd.read_csv(filename) # You can also add parameters such as header, sep, etc.
array = data.values


data is a dataframe type, and uses values to get ndarray. You can also get a list by using array.tolist().

data

array.tolist()

You can easily do it by the following piece of code:

lines = open(filePath).readlines()


Introduced in Python 3.4, pathlib has a really convenient method for reading in text from files, as follows:

pathlib

from pathlib import Path
p = Path('my_text_file')
lines = p.read_text().splitlines()


(The splitlines call is what turns it from a string containing the whole contents of the file to a list of lines in the file).

splitlines

pathlib has a lot of handy conveniences in it. read_text is nice and concise, and you don't have to worry about opening and closing the file. If all you need to do with the file is read it all in in one go, it's a good choice.

pathlib

read_text

You could also use the loadtxt command in NumPy. This checks for fewer conditions than genfromtxt, so it may be faster.

import numpy
data = numpy.loadtxt(filename, delimiter="n")


#!/bin/python3
import os
import sys
abspath = os.path.abspath(__file__)
dname = os.path.dirname(abspath)
filename = dname + sys.argv[1]
arr = open(filename).read().split("n")
print(arr)


python3 somefile.py input_file_name.txt


I like to use the following. Reading the lines immediately.

contents =
for line in open(filepath, 'r').readlines():
contents.append(line.strip())


Or using list comprehension:

contents = [line.strip() for line in open(filepath, 'r').readlines()]


With a filename, handling the file from a Path(filename) object, or directly with open(filename) as f, do one of the following:

filename

Path(filename)

open(filename) as f

list(fileinput.input(filename))

with path.open() as f

f.readlines()

list(f)

path.read_text().splitlines()

path.read_text().splitlines(keepends=True)

fileinput.input

f

list.append

f

list.extend

f

I explain the use-case for each below.

This is an excellent question. First, let's create some sample data:

from pathlib import Path
Path('filename').write_text('foonbarnbaz')


File objects are lazy iterators, so just iterate over it.

filename = 'filename'
with open(filename) as f:
for line in f:
line # do something with the line


Alternatively, if you have multiple files, use fileinput.input, another lazy iterator. With just one file:

fileinput.input

import fileinput

for line in fileinput.input(filename):
line # process the line


or for multiple files, pass it a list of filenames:

for line in fileinput.input([filename]*2):
line # process the line


Again, f and fileinput.input above both are/return lazy iterators.
You can only use an iterator one time, so to provide functional code while avoiding verbosity I'll use the slightly more terse fileinput.input(filename) where apropos from here.

f

fileinput.input

fileinput.input(filename)

Ah but you want it in a list for some reason? I'd avoid that if possible. But if you insist... just pass the result of fileinput.input(filename) to list:

fileinput.input(filename)

list

list(fileinput.input(filename))


Another direct answer is to call f.readlines, which returns the contents of the file (up to an optional hint number of characters, so you could break this up into multiple lists that way).

f.readlines

hint

You can get to this file object two ways. One way is to pass the filename to the open builtin:

open

filename = 'filename'

with open(filename) as f:
f.readlines()


or using the new Path object from the pathlib module (which I have become quite fond of, and will use from here on):

pathlib

from pathlib import Path

path = Path(filename)

with path.open() as f:
f.readlines()


list will also consume the file iterator and return a list - a quite direct method as well:

list

with path.open() as f:
list(f)


If you don't mind reading the entire text into memory as a single string before splitting it, you can do this as a one-liner with the Path object and the splitlines() string method. By default, splitlines removes the newlines:

Path

splitlines()

splitlines

path.read_text().splitlines()


If you want to keep the newlines, pass keepends=True:

keepends=True

path.read_text().splitlines(keepends=True)


I want to read the file line by line and append each line to the end of the list.

Now this is a bit silly to ask for, given that we've demonstrated the end result easily with several methods. But you might need to filter or operate on the lines as you make your list, so let's humor this request.

Using list.append would allow you to filter or operate on each line before you append it:

list.append

line_list =
for line in fileinput.input(filename):
line_list.append(line)

line_list


Using list.extend would be a bit more direct, and perhaps useful if you have a preexisting list:

list.extend

line_list =
line_list.extend(fileinput.input(filename))
line_list


Or more idiomatically, we could instead use a list comprehension, and map and filter inside it if desirable:

[line for line in fileinput.input(filename)]


Or even more directly, to close the circle, just pass it to list to create a new list directly without operating on the lines:

list(fileinput.input(filename))


You've seen many ways to get lines from a file into a list, but I'd recommend you avoid materializing large quantities of data into a list and instead use Python's lazy iteration to process the data if possible.

That is, prefer fileinput.input or with path.open() as f.

fileinput.input

with path.open() as f

Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).



Would you like to answer one of these unanswered questions instead?