Snippets are tiny notes I've collected for easy reference.
Backup or mirror a website using wget
To create a local mirror or backup of a website with wget
, run:
wget -r -l 5 -k -w 1 --random-wait <URL>
Where:
-r
(or--recursive
) will causewget
to recursively download files-l N
(or--level=N
) will limit recursion to at most N levels below the root document (defaults to 5, useinf
for infinite recursion)-k
(or--convert-links
) will causewget
to convert links in the downloaded documents so that the files can be viewed locally-w
(or--wait=N
) will causewget
to wait N seconds between requests--random-wait
will causewget
to randomly vary the wait time to0.5x
to1.5x
the value specified by--wait
Some additional notes:
--mirror
(or-m
) can be used as a shortcut for-r -N -l inf --no-remove-listing
which enables infinite recursion and preserves both the server timestamps and FTP directory listings.-np
(--no-parent
) can be used to limitwget
to files below a specific "directory" (path).
Published 10 Feb 2014
Snippets are tiny notes I've collected for easy reference.