Web-Archiving: Collecting and Preserving Important Web-based National Resources

Transcription

1 Web-Archiving: Collecting and Preserving Important Web-based National Resources Mark Phillips Dr. Daniel Gelaw Alemneh University of North Texas UNT Libraries

2 The Web is the platform for communication for the 21st century. 2

3 Each year more and more content is created digitally. 3

4 And published to the world via the Web. 4

5 Ensuring the long-term preservation and access to these resources offer new challenges and opportunities to memory organizations such as libraries, museums, and archives. 5

6 Background: 6

7 Web archiving is the process of collecting portions of the World Wide Web to ensure the information is preserved in an archive for future researchers, historians, and the public. 7

8 It involves concepts that are core to libraries and archives. 8

9 Selection: Defining the scope of the Web for preservation. 9

10 National Domain Federal Domain State/Municipal Web Subject Based Event Based 10

11 Acquisition: Harvesting selected content by using Web crawlers such as Heritrix 11

12 Access: Enabling interested users to accessing Web resources from the past. 12

13 Preservation: Ensuring the long-term access to these Web resources through systematic planning and curation. 13

14 These concepts enable memory organizations to align Web archiving with traditional collections, processes and missions 14

15 Examples: 15

16 UNT Libraries CyberCemetery 16

17 17

18 Archive of Federal agencies, commissions, committees and initiatives that are no longer active. 18

19 Started in 1997 with the Advisory Commission on Intergovernmental Relations (ACIR) 19

20 Today contains over 100 Websites 20

21 UNT Libraries is an Affiliated Archives with the U.S. National Archives and Records Administration (NARA) because of the CyberCemetery 21

22 End of Term Presidential Web Archives 2008, 2012,

23 23

24 24

25 25

26 26

27 27

28 Collaboration between institutions throughout the United State to capture the Federal (.gov) Web at the end of each presidential term. 28

29 Selection, harvesting, archiving distributed to meet local collection needs. 29

30 Centralized access 30

31 UNT Libraries has acquired archive for 2008 and 2012 resulting in a 16TB and 30TB dataset 31

32 Extracted.gov dataset from Internet Archive 32

33 Collaboration between UNT and the Internet Archive in 2014 to extract the.gov data from the global Wayback instance at the Internet Archive 33

34 109 TB dataset 34

35 Using Web Archives in scholarship, teaching and research 35

36 Collaborating Internationally 36

37 International Internet Preservation Consortium (netpreserve.org) 37

38 38

39 51 member institutions 39

40 Members from 25 countries: North America, Europe, Asia, Africa 40

41 National Libraries National Archives Research Libraries Government Agencies Private Companies 41

42 Common goal to preserve the Web 42

43 Collaboratively develop tools for the community such as the Heritrix Web crawler and the Open Wayback system for replay of Web archives. 43

44 44

45 Lead work on the formalization of the WARC File Format (ISO 28500:2009) 45

46 Web archiving is an important component in preserving a national heritage for future generations. 46

47 Thank you. አመሰግናለሁ Mark Phillips Dr. Daniel Gelaw Alemneh 47