LiWA technologies released in Open Source
LiWA partners are pleased to announce the release in open-source of the complete list of components and tools issued from the LiWA project.
They are all grouped under the “liwa-technologies” project on Google code:
http://code.google.com/p/liwa-technologies/.
1° The Rich Media Capture Module - a plug-in dedicated to the capture of streaming video content:
http://code.google.com/p/liwa-technologies/source/browse/rich-media-capture
http://code.google.com/p/liwa-technologies/downloads/detail?name=rich-media-capture-plugin-1.0.jar
2° The Temporal Coherence Analyser - a plug-in dedicated to the analysis of the temporal coherence of the archived Web content:
http://code.google.com/p/liwa-technologies/source/browse/temporal-coherence
3° The Spam Assessment Interface - a Web service that enables the quality assessment of the archived Web content:
http://code.google.com/p/liwa-technologies/source/browse/assessment-interface
4° The Semantic Analizer - a component dedicated to the detection of terminology evolution:
http://code.google.com/p/liwa-technologies/source/browse/SemanticAnalyser
http://code.google.com/p/liwa-technologies/downloads/detail?name=SemanticAnalyser-1.0.zip
5° The Web Archive UI Framework - a client-side framework that helps creating User Interface helpers for Web archive browsing:
http://code.google.com/p/liwa-technologies/source/browse/web-archive-ui-framework
To learn more about each component, the Google project provides also a wiki space, giving a brief description of each module and the necessary steps for its deployment: http://code.google.com/p/liwa-technologies/w/list
You are all welcome to download and try out the LiWA components. Your feedback and comments will be greatly appreciated, helping us to improve the documentation and the usability of the technologies.
Terminology Extraction Pipeline Version 1.0 released
The LiWA consortium released the Terminology Extraction Pipeline Version 1.0 as part of the Terminology Evolution Detection module. The released processing pipeline consists of four major steps: pre-processing, natural language processing and creation of co-occurrence graph.
In more detail the WARC Collection Reader (WARC Extraction) extracts the text and time metadata for each site archived in the input crawl. The POS (Part Of Speech) Tagger is an aggregate analysis engine from Dextract . It consists of a tokenizer, a language independent part of speech tagger and lemmatizer (TreeTagger). In the Term Extraction sub-module, we read the annotated sites, extract the lemmas and the different occurring parts of speech that were identified for the archived sites. After that, we index the terms in an database (MySQL) index (see below). In the Co-occurrence Analysis we extract lemma or noun co-occurrence matrices for the indexed crawl from the database index.
The Terminology Extraction Pipeline can be downloaded from Google Code:
http://code.google.com/p/liwa-technologies/downloads/detail?name=SemanticAnalyser-1.0.zip
LiWA Project listed on ePractice.eu
ePractice.eu is a new service that merges the eGovernment Observatory with the Good Practice Framework and allows to meet people, share experience and learn in public e-domains. The LiWA Project is now listed in the cases of the eGovernment domain .
LiWA technologies first release
LiWA partners have been busy the last 2 years developing new Web Archiving technology within the LiWA project.
They are now pleased to announce the release in open-source of several components, as part of the “liwa-technologies” project on Google code: http://code.google.com/p/liwa-technologies/.
You are all welcome to download and try out the plug-in dedicated to the streaming video content, the Rich Media Capture module:
http://code.google.com/p/liwa-technologies/source/browse/rich-media-capture
http://code.google.com/p/liwa-technologies/downloads/detail?name=rich-media-capture-plugin-1.0.jar
This is still an experimental version of the software, therefore your feedback and comments will be greatly appreciated, helping us to improve the documentation and the usability of the module.
More LiWA tools will be available soon, please stay tuned for the next releases.
Using Word Sense Discrimination on Historic Document Collections
The paper “Using Word Sense Discrimination on Historic Document Collections” has been accepted for the 10th ACM/IEEE JCDL
The paper entitled “Using Word Sense Discrimination on Historic Document Collections” by Nina Tahmasebi, Kai Niklas, Thomas Theuerkauf and Thomas Risse has been accepted in the 10th ACM/IEEE Joint Conference on Digital Libraries. The paper evaluates word sense discrimination on historic document collections to investigate if word senses can be found automatically using modern technology applied on historic data. The paper also investigates which impact OCR errors, present in scanned historic documents, have on finding word senses in an automatic way. Finding word senses in an automatic way is the first step towards detecting terminology evolution and hence an important step in our research. Nina Tahmasebi will present the paper on June 22nd, 2010 at JCDL which is held in conjunction with ICADL in Surfers Paradise (Gold Coast, Australia).
Update on LiWA Project to GI Working Group
Thomas Risse presented Liwa at the meeting the GI Working Group on Digital Libraries
Thomas Risse presented the members of the GI Working Group on Digital Libraries during their bi-annual meeting on Tuesday, 14.4.2010 an update on the activities and results of the LiWA project.
IWAW Proceedings online
IWAW09 took take place the 30th of September and 1st of October 2009, in conjunction with ECDL in Corfu (Greece). The proceedings are now available online.
Around 40 participants attended IWAW2009, which took place on Sep. 30 / Oct. 1 2009, in conjunction with ECDL in Corfu (Greece). The workshop provided a comprehensive overview on active research and practice on the preservation of the Web. This year’s workshop also addressed several new approaches and research (from virtual worlds preservation to temporal dimension of Web Archives) as well as practical issues addressed by Archiving institutions, specifically with respect to managing the storage of large volumes of digital material. In this context, a special Session was devoted to the WARC storage format, which has been accepted as a new ISO standard (ISO 28500:2009), as well as emerging tool support to handle these container objects. In general, scalability issues and managing large-volume crawls were topics of intensive discussions, based on the increasing body of experience available in numerous institutions by now, running a series of Web archiving activities in a range of different configurations.
Talk about “Turning pure Web Page Storages into Living Web Archives” at Cultural Heritage on line
The LiWA applications and its R&D challenges will be presented at the Conference “Cultural Heritage on line Empowering users: an active role for user communities” at Florence, Italy on the 15th and 16th of December, 2009
Web content plays an increasingly important role in the knowledge-based society, and the preservation and long-term accessibility of Web history has high value (e.g., for scholarly studies, market analyses, intellectual property disputes, etc.). There is strongly growing interest in its preservation by libraries and archival organizations as well as emerging industrial services. Web content characteristics (high dynamics, volatility, contributor and format variety) make adequate Web archiving a challenge.
LiWA will look beyond the pure “freezing” of Web content snapshots for a long time, transforming pure snapshot storage into a “Living” Web Archive. In order to create Living Web Archives, the LiWA project will address R&D challenges in the three areas: Archive Fidelity, Archive coherence and Archive interpretability. The results of the project will be demonstrated within two application scenarios namely “Streaming Archive” and “Social Web Archive”. The Streaming Archive application will showcase the building of an audio-visual Web archive and how audio and video broadcast related web information can be preserved. The Social Web application will demonstrate how web archives can capture the dynamics and the different types of user interaction of the social web.
LiWA Second Newsletter published
The LiWA Newsletter No2 is now available.
For this second Newsletter, focus has been given on Streaming and Social Web applications.
Learn more by reading the presentation of the LiWA partners involved in the development of each application.
Talk “From Web page storages to Living Web Archive” at London
Dr. Thomas Risse (L3S) will give a talk at the “JISC, the DPC and the UK Web Archiving Consortium Workshop”, at the The British Library Conference Centre in London, on July 21st.
The paper on “From Web page storages to Living Web Archive” will be presented by Dr. Thomas Risse, at the JISC, the DPC and the UK Web Archiving Consortium Workshop which will take place at The British Library Conference Centre in London, on July 21st.
Bridging the Terminology Gap in Web Archive Search
A paper on dealing with terminology evolution in web archives has been accepted in the 12th International Workshop on the Web and Databases (WebDB 2009)
The paper entitled ’Bridging the Terminology Gap in Web Archive Search’ by Klaus Berberich, Srikanta Bedathur, Mauro Sozio, and Gerhard Weikum has been accepted in the 12th International Workshop on the Web and Databases (WebDB 2009). The paper proposes a method to find query reformulations that paraphrase users’ information needs using past terminology. Such query reformulations are key to retrieving old but highly relevant documents in web archives that were written using now outdated terminology. Klaus Berberich will present the paper on June 28th, 2009 at WebDB 2009, which is held in conjunction with SIGMOD 2009 in Providence (Rhode Island, USA).
SHARC: Framework for Quality-Conscious Web Archiving
A paper on quality-conscious web archiving has been accepted in the 35th International Conference on Very Large Data Bases (VLDB 2009)
The paper on quality-conscious web archiving by Dimitar Denev, Arturas Mazeika, Marc Spaniol, and Gerhard Weikum has been accepted for presentation to the 35th International Conference on Very Large Data Bases (VLDB 2009). The conference takes place on 24-28 August in Lyon, France. The paper presents the SHARC framework for assessing the data quality in Web archives and for tuning capturing strategies towards better quality with given resources. The paper defines quality measures, characterise their properties, and derives a suite of quality-conscious scheduling strategies for archive crawling.
LiWA Poster at the CHORUS Conference in Brussels
Nina Tahmasebi (L3S) will present a poster on the LiWA project in the CHORUS conference in Brussels (Belgium) on May 26-27, 2009. The poster will present the overall goals of LiWA and in particular semantic evolution.
Talk “Terminology Evolution in Web Archiving” at Vienna
Dr. Thomas Risse (L3S) will give a talk on “Terminology Evolution in Web Archiving” at Seminarraum, Institut für Knowledge und Business Engineering, in Rathausstrasse 19/9, A-1010 Wien; on April 30, 2009.
Due to the central role that the World Wide Web plays in nearly all areas
of today’s life, adequate Web archiving has become a cultural necessity in
preserving knowledge. The next generation web archiving technologies will
overcome limitations in content capture, preservation, analysis and
enrichment. One important aspect is the archive interpretability. The
correspondence between the terminology used for querying and the one used
in content objects to be retrieved is a crucial prerequisite for effective
content access based on retrieval technology. However, as terminology is
evolving over time, a growing gap opens up between older documents in
(long-term) archives and the active language used for querying such
archives. Thus, technologies for detecting and systematically handling
terminology evolution are required to ensure ``semantic’’ accessibility of
(Web) archive content on the long run. As a starting point for dealing
with terminology evolution present the problem and discusses issues,
approaches and relevant technologies.
Talk “Data Quality in Web Archiving”
A talk on “Data Quality in Web Archiving” will be given at the 3rd Workshop on Information Credibility on the Web (WICOW 2009) in Madrid (Spain) on Monday, April 20.
The paper on “Data Quality in Web Archiving” by Marc Spaniol, Dimitar Denev, Arturas Mazeika, Pierre Senellart and Gerhard Weikum will be presented at the 3rd Workshop on Information Credibility on the Web (WICOW 2009). The Workshop and paper presentation takes places on April 20 at Madrid, Spain and is organized in conjunction with the 18th International World Wide Web Conference (WWW 2009). The paper addresses the problems of capturing a large Web site that may span hours or even days, which increases the risk that contents collected so far are incoherent with the parts that are still to be crawled. The paper introduces a model for identifying coherent sections of an archive and, thus, measuring the data quality in Web archiving. Additionally, a crawling strategy is introduced that aims to ensure archive coherence by minimizing the diffusion of Web site captures.
Talk “Building on Internet Memory” at Grenoble Xerox Research Centre Europe (XRCE)
Julien Masanès (EA) gave a talk on “Building on Internet Memory” during a XRCE Seminar in Grenoble on March 12, 2009.
LiWA has been presented, as well as the necessity and challenges of the digital preservation, in the current context of mass information, diversity and evolution of the web.
Lecture “Web Archiving” at Stuttgart Media University (HdM) on March 17, 2009
Marc Spaniol (Max-Planck-Institute for Computer Science) gave a lecture on “Web Archiving” within the scope of the course “Creation of an E-learning Module for Web Archiving” on March 17, 2009 at Stuttgart Media University (HdM), Germany.
On March 17, 2009 Dr. Marc Spaniol from Max-Planck-Institute for Computer Science has presented ongoing research taking place within the LiWA project to students and scientists of Stuttgart Media University (HdM). During the 90 minutes lecture, Dr. Spaniol introduced the main issues in Web archiving and present examples of Web spam as well as temporal coherence ensuring crawling strategies to the audience. The lecture was part of an elective course on “Creation of an E-learning Module for Web Archiving” in the scope of the Library and Information Management program. This course is organized by Prof. Markus Hennies and Prof. Heidrun Wiesenmüller M.A.. The lecture is open to public and takes place at 14.15 on March 17 at HdM’s site in Wolframstraße. Guests interested in Web archiving and/or LiWA (in particular) are welcome.
LiWA first newsletter published
The first edition of LiWA NEWS, our newsletter, is published.
For the first edition of LiWA NEWS, our newsletter, we have asked all the LiWA research partners to present their goals and summarize their achievements for the first year. We hope you will enjoy reading it.
Lecture “Web Archiving” held at RWTH Aachen University
Marc Spaniol (Max-Planck-Institute for Computer Science) gave a lecture on “Web Archiving” within the scope of the “Web Science” course on November 28, 2008 at RWTH Aachen University, Germany.
The course on Web Science at RWTH Aachen University organized by Prof. Dr. Matthias Jarke and Dr. Ralf Klamma of Lehrstuhl Informatik 5 addresses Web Science as a new and challenging study field in computer science. This course covers the wide range of current and emerging Web concepts, technologies and Web-based software systems. In order to present recent approaches in the scope of Web archiving, Dr. Marc Spaniol from Max-Planck-Institute for Computer Science was invited to present ongoing research taking place within the LiWA project. During the 90 minutes lecture, Dr. Spaniol introduced the main issues in Web archiving and presented examples of Web spam as well as temporal coherence ensuring crawling strategies to the audience.
Half day session on LiWA during IWAW
A dedicated session took place during the 8th International Web Archiving Workshop
Over 70 web archivists and researchers in this domain attended the 8th edition of IWAW during which a full session was dedicated to present research objectives and early results from LiWA.
Lots of questions and interest from the audience, which is good sign for us. See below links to presentations from this session:
Web Spam: a Survey with Vision for the Archivist
Andras Benczur, David Siklosi, Jacint Szabo, Istvan Biro, Zsolt Fekete, Miklos Kurucz, Attila Pereszlenyi, Simon Racz, Adrienn Szabo (paper, presentation)
Terminology Evolution in Web Archiving: Open Issues
Nina Tahmasebi, Tereza Iofciu, Thomas Risse, Claudia Niederée, Wolf Siberski (paper,presentation)
Liwa Architecture
Radu Pop, Wolf Siberski, Mark Williamson (presentation)
“Catch me if you can”. Temporal Coherence of Web Archives
Marc Spaniol (presentation)
The Challenge of Dynamic Links
Mark Williamson (presentation)
Presentation at the IFIP WG 2.6. Meeting
Thomas Risse (L3S) presented LiWA and the problem of terminology evolution during the meeting of the IFIP 2.6 Working Group on Databases
Due to the central role that the World Wide Web plays in nearly all areas of today’s life, adequate Web archiving has become a cultural necessity in preserving knowledge. The next generation web archiving technologies will overcome limitations in content capture, preservation, analysis and enrichment. One important aspect is the archive interpretability. The correspondence between the terminology used for querying and the one used in content objects to be retrieved is a crucial prerequisite for effective content access based on retrieval technology. However, as terminology is evolving over time, a growing gap opens up between older documents in (long-term) archives and the active language used for querying such archives. Thus, technologies for detecting and systematically handling terminology evolution are required to ensure ``semantic’’ accessibility of (Web) archive content on the long run. As a starting point for dealing with terminology evolution present the problem and discusses issues, first ideas and relevant technologies.
Presentation at the GI-DL Meeting in Karlsruhe
Thomas Risse (L3S) presented LiWA and the problem terminology evolution during the foundation meeting of the German Digital Library Working Group of the Gesellschaft für Informatik e.V.
Presentation at the University Stuttgart
Thomas Risse (L3S) presented LiWA and the problem terminology evolution at the Institute for Natural Language Processing of the University Stuttgart
Abstract
Due to the central role that the World Wide Web plays in nearly all areas of today’s life, adequate Web archiving has become a cultural necessity in preserving knowledge. A first generation of Web archiving technology has been built by pioneers in the domain based on existing search technology. The next generation web archiving technologies will overcome limitations in content capture, preservation, analysis and enrichment. It is the goal of the LiWA project (Living Web Archives, IST FP7 216267) to turn Web archives from pure Web page storages into “living Web archives”. Such living archives, will be capable of: handling a variety of content types; dealing with evolution as well as long-term archive interpretability.
One important aspect is the archive interpretability. The correspondence between the terminology used for querying and the one used in content objects to be retrieved is a crucial prerequisite for effective content access based on retrieval technology. However, as terminology is evolving over time, a growing gap opens up between older documents in (long-term) archives and the active language used for querying such archives. Thus, technologies for detecting and systematically handling terminology evolution are required to ensure ``semantic’’ accessibility of (Web) archive content on the long run.
Within this talk we give an overview about the LiWA project and present in more detail the problem of terminology evolution by giving a more formal problem statement and discuss issues, first ideas and relevant technologies.
Living Web Archives Announcement
Press Release
February 18, 2008
The Living Web Archives project will carry Web archiving beyond the current approach, characterized by static snapshots, to one that fully accounts for the dynamics and interrelations of Web content. Through an ambitious plan of research, the project aims to improve archive fidelity, coherence, and interpretability.
The three-year project is funded by the European Union through the Seventh Research Framework Programme.
The project kick-off meeting was held earlier this month in Hannover, Germany.
The result of LiWA’s work will be a set of next generation Web archiving methods and tools making possible the creation and long-term usability of high-quality Web archives. Aspects of the project’s research will focus on improving Web capture fidelity by providing for the capture of the hidden Web and all types of content, including content that is currently difficult to gather, and by filtering out unwanted content through spam and trap detection.
Other aspects of the project will address the temporal incoherence inherent in current Web capture methods and tools. The research will also address the rapid semantic and technological evolution of the Web in order to promote the long-term viability of Web archives.
LiWA research will lead to scalable technologies, applicable to the Web archiving goals in institutions with varying collection policies.
- European Archive Foundation, The Netherlands
- Max Planck Institut for Computer Science, Germany
- Computer and Automation Research Institute of the Hungarian Academy of Sciences, Hungary
- Netherlands Institute for Sound & Vision, The Netherlands
- Hanzo Archives Limited, England
- National Library of the Czech Republic, Czech Republic
- Moravian Library, Czech Republic
For more information on Living Web Archives, please visit: http://www.liwa-project.eu/index.html
Project Presentation at the Information Day
December 17, 2007 Luxemburg
Download the slides : [pdf 1.30Mb]
