CLARIN-NL - corpus processing

Institute for the Dutch Language

Project Contact

dr. Martin Reynaert

Creator Contact

Dr. Arianna Betti

Documentation

Usage

Source code

not specified

Original source

http://portal.clarin.nl/node/4190

Publications

Betti, A, Reynaert, M and van den Berg, H. 2017. @PhilosTEI: Building Corpora for Philosophers. In: Odijk, J and van Hessen, A. (eds.) CLARIN in the Low Countries, Pp. 379–392. London: Ubiquity Press. DOI: https://doi.org/10.5334/bbi.32. License: CC-BY 4.0

Resource

CMDI File Link

License

unknown

Inventory Scope

local

VU-DNC

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

VU University Diachronic News text Corpus

Description

The diachronic corpus has been brought in line with current standards and formats as used in the STEVIN Nederlandstalig Referentiecorpus (SoNaR, under development), which has been adapted to the more general FoLiA format (documented by Van Gompel, 2012). These standards and formats have been extended with new layers of annotation. As a result the corpus adheres to the current day CLARIN infrastructure.

Project

VU-DNC

CLARIN National Project

CLARIN-NL

CLARIN centre

Dutch Language Institute

Research domain

Linguistics

Linguistic Subject

Computational Linguistics

general linguistics

Lexicology

Morphology

Syntax

discourse analysis

text and corpus linguistics

Tool task

corpus browsing

corpus searching

Country

Netherlands

Tool Type

web application

Research Phase

Tool status

published

Input Language

Dutch

Access Contact

Institute for the Dutch Language

Project Contact

/mailto:-/

Creator Contact

Dr. Kisten Vis

dr. M.W.C. (Martin) Reynaert

Laura van Eerten

Institute for the Dutch Language

RU Nijmegen

Source code

not specified

Original source

http://portal.clarin.nl/node/4194

Resource

SearchPage

LandingPage

CMDI File Link

License

other

Inventory Scope

local

TTNWW

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

TTNWW - TST Tools for the Dutch Language as Web services in a Workflow

Description

TTNWW integrates and makes available existing Language Technology (LT) software components for the Dutch language that have been developed in the STEVIN and CGN projects. The LT components (for text and speech) are made available as web-services in a simplified workflow system that enables researchers without much technical background to use standard LT workflow recipes. The web services are available in two separate domains: "Text" and "Speech" processing. For "Text", workflows for the following functionality is offered by TTNWW: - Orthographic Normalisation using TICCLops (version CLARIN-NL 1.0); - Part of Speech Tagging, Lemmatisation, Chunking, limited Multiword Unit Recognition, and Grammatical Relation Assignment by Frog (Version 012.012); - Syntactic Parsing (including grammatical relation assignment, limited named entity recognition, and limited multiword unit recognition) by the Alpino Parser (version 1.3); - Semantic Annotation; - Named Entity Recognition; - Co-reference Assignment. For "Speech", the following workflows are offered: - Automatic Transcription of speech files using a Netherlands Dutch acoustic model; - Automatic Transcription of speech files using a Flemish Dutch acoustic model; - Conversion of the input speech file to the required sampling rate, followed by automatic transcription. The TTNWW services have been created in a Dutch and Flemish collaboration project building on the results of past Dutch and Flemish projects. The web services are partly deployed in the SURF-SARA BiG-Grid cloud or at CLARIN centres in the Netherlands and at CLARIN VL University partners. The architecture of the TTNWW portal consists out of several components and follows the principles of Service Oriented Architecture (SOA). The TTNWW GUI front-end is a Flex module that communicates with the TTNWW web-application which keeps track of the different sessions and knows which LT recipes are available. TTNWW communicates assigments (workflow specifications) to the WorkflowService that evaluates the requested workflow and requests the DeploymentSevice to start the required LT web-services. After initialization of the LT web-services, the workflow specification is sent to the Taverna Server, that takes further care of the workflow. To facilitate the process of wrapping applications that were originally designed as standalone applications into web services, the CLAM (Computational Linguistics Application Mediator) wrapper software allows for easy and transparent transformation of applications into RESTful web services. The CLAM software has extensively been used in the TTNWW project for both text and speech processing tools. With the exception of Alpino and MBSRL all web services work operate on CLAM wrappers. Given the number of web services involved in the TTNWW project and possibilities offered by the cloud environment the preferred method of delivering the web service installations was delivery of complete virtual machine images by the LT providers. These could be directly uploaded into the cloud environment and thus relieving the CLARIN centres nd LT providers from the original foreseen task of running the webservices themselves. A potential advantage of this method, that has not been exploited in the project yet, is that these images may be also be delivered directly to the end user so these can be run in a local configuration using virtualization software such as VMWare of VirtualBox. The workflow engine used in the project was Taverna. But build on top of this was a a number of selectable task recipes, following a task oriented approach in line with the premises that users with no or little technical expertise should be able to use the system. In this context, tasks are understood in terms of end results of processes such as semantic role labelling, pos tagging or syntactic analysis and ready-made workflows are constructed that can be readily used by the end user.

Project

TTNWW

CLARIN National Project

CLARIN-NL

CLARIN centre

Meertens/HuC

Research domain

Linguistics

Communication and Media Studies

History

Oral History

Linguistic Subject

discourse analysis

Orthography

Semantics

Syntax

Tool task

grammatical relation assignment

coreference resolution

lemmatisation

multiword unit identification

named entity recognition

orthographic normalisation

part of speech tagging

semantic role labeling

chunking

parsing

speech recognition

speech transcription

tokenisation

up/down sampling

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

withdrawn

Input format

text/plain

audio/wav

Output format

text/xml

Input Language

Dutch

Access Contact

Daan Broeder

Project Contact

/mailto:Daan.Broeder@meertens.knaw.nl/

Creator Contact

/mailto:Daan.Broeder@meertens.knaw.nl/

Documentation

TTNWW handleiding

Source code

not specified

Original source

http://portal.clarin.nl/node/1964

Publications

Kemps-Snijders, M, Schuurman, I, Daelemans, W, Demuynck, K, Desplanques, B, Hoste, V, Huijbregts, M, Martens, J-P, Paulussen, H, Pelemans, J, Reynaert, M, Vandeghinste, V, van den Bosch, A, van denHeuvel, H, van Gompel, M, van Noord, G and Wambacq, P. 2017. TTNWW to the Rescue: No Need to Know How to Handle Tools and Resources. In: Odijk, J and van Hessen, A. (eds.) CLARIN in the Low Countries, Pp. 83–93. London: Ubiquity Press. DOI: https://doi.org/10.5334/bbi.7. License: CC-BY 4.0

Resource

CMDI File Link

License

unknown

Inventory Scope

local

TiCClops

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

TiCClops: Text-Induced Corpus Clean-up online processing system

Description

TICCL (Text Induced Corpus Clean-up) is a system that is designed to search a corpus for all existing variants of (potentially) all words occurring in the corpus. This corpus can be one text, or several, in one or more directories, located on one or more machines. TICCL creates word frequency lists, listing for each word type how often the word occurs in the corpus. These frequencies of the normalized word forms are the sum of the frequencies of the actual word forms found in the corpus. TICCL is a system that is intended to detect and correct typographical errors (misprints) and OCR errors (optical character recognition) in texts. When books or other texts are scanned from paper by a machine, that then turns these scans, i.e. images, into digital text files, errors occur. For instance, the letter combination `in' can be read as `m', and so the word `regeering' is incorrectly reproduced as `regeermg'. TICCL can be used to detect these errors and to suggest a correct form. Text-Induced Corpus Clean-up (TICCL) was developed first as a prototype at the request of the Koninklijke Bibliotheek - The Hague (KB) and reworked into a production tool according to KB specifications (currently at production version 2.0) mainly during the second half of 2008. It is a fully functional environment for processing possibly very large corpora in order to largely remove the undesirable lexical variation in them. It has provisions for various input and output formats, is flexible and robust and has very high recall and acceptable precision. As a spelling variation detection system it is to the developer’s knowledge unique in making principled use of the input text as possible source for target output canonical forms. As such it is far less domain-sensitive than other approaches: the domain is largely covered by the input text collection. TICCL comes in two variants: one with a classic CLAM web application interface, and one with the PhilosTEI interface.

Project

TiCClops: Text-Induced Corpus Clean-up online processing system

CLARIN National Project

CLARIN-NL

CLARIN centre

Dutch Language Institute

Research domain

Linguistics

Linguistic Subject

general linguistics

Orthography

Tool task

orthographic normalisation

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

published

Input format

image/tiff

image/vnd.djvu

application/pdf

text/plain

text/xml

text/csv

Output format

text/xml

Input Language

Dutch

English

Finnish

French

German

German (Fraktur)

Classical Greek

Modern Greek

Icelandic

Italian

Latin

Polish

Portuguese

Russian

Spanish

Swedish

Access Contact

Institute for the Dutch Language

Project Contact

dr. Martin Reynaert

Creator Contact

dr. Martin Reynaert

Documentation

TICCLops User and Demonstrator Documentation

Source code

not specified

Original source

http://portal.clarin.nl/node/1914

Publications

Reynaert, M. (2008). All, and only, the errors: More complete and consistent spelling and OCR-error correction evaluation. In: Proceedings of the Sixth International Language Resources and Evaluation (LREC’08), Marrakech, Morocco.

Reynaert, M. (2010). Character confusion versus focus word-based correction of spelling and ocr variants in corpora. International Journal on Document Analysis and Recognition, pp 1-15, URL http://dx.doi.org/10.1007/s10032-010-0133-5

Resource

CMDI File Link

License

unknown

Inventory Scope

local

RemBench

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

RemBench - a Digital Workbench for Rembrandt Research

Description

RemBench enables one to search and browse for works of art, artists, primary sources and library sources related to Rembrandt, using faceted search by location, author/artist name, author/artist type, and date range, and/or by both exact and fuzzy keyword search. It offers both a web application and a RESTful web service. RemBench combines the content of four different databases behind one search interface: RKDartists and RKDimages, two databases maintained by the Netherlands Institute for Art History (RKD); RemDoc, a collection of original documents related to Rembrandt van Rijn from the period between 1475 to circa 1750; RUQuest, a library system that provides access to full text articles, as well as the complete collection of (e-)books and journals from the Radboud University Library Catalogue. RemBench does not influence the content of these databases.

Project

RemBench

CLARIN National Project

CLARIN-NL

CLARIN centre

Huygens ING

Research domain

History

Art History

Tool task

corpus searching

Country

Netherlands

Tool Type

web application

Research Phase

RemBench, a Digital Workbench for Rembrandt Research - User guidelines

Tool status

released

Input Language

Dutch

English

Version

0.2.1

Access Contact

Guido Gerritsen

Project Contact

Guido Gerritsen

Creator Contact

Suzan Verberne

Documentation

Technical Documentation RemBench

Source code

not specified

Original source

http://portal.clarin.nl/node/4177

Publications

Verberne, S, van Leeuwen, R, Gerritsen, G and Boves, L. 2017. RemBench: A Digital Workbench for Rembrandt Research. In: Odijk, J and van Hessen, A. (eds.) CLARIN in the Low Countries, Pp. 337–350. London: Ubiquity Press. DOI: https://doi.org/10.5334/bbi.28. License: CC-BY 4.0

Resource

CMDI File Link

License

unknown

Inventory Scope

local

PaQu

root — Mon, 08 Apr 2019 08:38:55 +0000

Title

PaQu - Parse and Query

Description

PaQu uses the Alpino parser to make treebanks of your own text corpus, and to search in these treebanks using an interface based on the LASSY Word Relations Search interface (http://dev.clarin.nl/node/1966). Several treebanks are already available in the application, such as: Lassy Klein (1M words, manually checked syntactic analysis) and Lassy Groot (700M words, syntactic analysis automatically assigned by Alpino). PaQu offers two ways to search through the syntactically annotated texts. The first option is to use the search bar to look for word pairs, optionally complemented by their syntactic relationship. The second search option is to use the query language XPath.

Project

PaQu

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Computational Linguistics

general linguistics

Syntax

Tool task

corpus browsing

corpus searching

parsing

grammatical relation assignment

Country

Netherlands

Tool Type

web application

Research Phase

Data analysis

Enriching Data

Tool status

published

Input format

text/plain

Output format

text/plain

Input Language

Dutch

Version

no explicit version mentioned

Access Contact

Gertjan van Noord

Project Contact

/mailto:g.j.m.van.noord@rug.nl/

Creator Contact

/mailto:p.c.j.kleiweg@rug.nl/

Documentation

PaQu Info

paqu/README.md

Source code

https://github.com/rug-compling/paqu

Original source

http://portal.clarin.nl/node/4182

Publications

Odijk, J, van Noord, G, Kleiweg, P and Tjong Kim Sang, E. 2017. The Parse and Query (PaQu) Application. In: Odijk, J and van Hessen, A. (eds.) CLARIN in the Low Countries, Pp. 281–297. London: Ubiquity Press. DOI: https://doi.org/10.5334/bbi.23. License: CC-BY 4.0

Resource

CMDI File Link

License

other

Inventory Scope

local

OpenConvert

root — Mon, 08 Apr 2019 08:38:48 +0000

Title

OpenConvert

Description

The OpenConvert tools convert to TEI or FOLiA from a number of input formats (alto, text, word, HTML, ePub). The tools are available as a Java command line tool, a web service and a web application.The OpenConvert Tools were created by IVDNT in the OpenConvert project. The OpenConvert tools convert to TEI or FOLiA from a number of input formats (alto, text, word, HTML, ePub). The tools are available as a Java command line tool, a web service and a web application. Furthermore, as a proof of concept, the website currently provides two annotation tools: a simple Tokenizer for TEI files and a modern Dutch part of speech tagger.

Project

OpenConvert

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

Dutch Language Institute

Research domain

Linguistics

Religion Studies

Communication and Media Studies

Cultural Sciences

History

Literary Studies

Philosophy

Political Studies

Tool task

format conversion

text conversion

tokenisation

part of speech tagging

Country

Netherlands

Tool Type

local desktop

Research Phase

Enriching Data

Tool status

released

Input format

text/plain

application/msword

text/html

text/xml

application/epub+zip

application/zip

Output format

text/xml

Input Language

Language independent

Access Contact

Institute for the Dutch Language

Project Contact

Jan Theo Bakker

Institute for the Dutch Language

Creator Contact

Jan Theo Bakker

Institute for the Dutch Language

Documentation

OpenConvert help

Source code

https://github.com/INL/OpenConvert

Original source

http://portal.clarin.nl/node/4224

Resource

CMDI File Link

License

other

Inventory Scope

local

Nederlab

root — Mon, 08 Apr 2019 08:38:48 +0000

Title

Nederlab, online laboratory for humanities research on Dutch text collections

Description

The Nederlab project aims to bring together all digitized texts relevant to Dutch national heritage, the history of Dutch language and culture (c. 800 - present) in one user-friendly and tool-enriched open access web interface, allowing scholars to simultaneously search and analyze data from texts spanning the full recorded history of the Netherlands, its language and culture. The project builds on various initiatives: for corpora Nederlab collaborates with the scientific libraries and institutions, for infrastructure with CLARIN (and CLARIAH), for tools with eHumanities programmes such as Catch, IMPACT and CLARIN (TICCL, frog). Nederlab will offer a large number of search options with which researchers can find the occurrence of a particular term in a particular corpus or subcorpus. It'll also offer visualization of search results through line graphs, bar graphs, circle graphs, or scatter graphs. Furthermore, this online lab will offer a large set of tools, like tokenization tools, tools for spelling normalization, PoS-tagging tools, lemmatization tools, a computational historical lexicon and indices. Also, the use of (semi-) automatic syntactic parsing, tools for text mining, data mining and sentiment mining, Named Entity Recognition tools, coreference resolution tools, plagiarism detection tools, paraphrase detection tools and cartographical tools is offered The first version of Nederlab was launched in early 2015, it’ll be expanded until the end of 2017. Nederlab is financed by NWO, KNAW, CLARIAH and CLARIN-NL.

Project

Nederlab

CLARIN National Project

CLARIN-NL

CLARIAH-CORE

CLARIN centre

Meertens/HuC

Research domain

Linguistics

History

Cultural Sciences

Linguistic Subject

historical linguistics

Tool task

coreference resolution

corpus searching

corpus workbench

lemmatisation

part of speech tagging

tokenisation

Country

Netherlands

Tool Type

web application

Research Phase

Data analysis

Tool status

published

Input Language

Dutch

Version

Access Contact

Hennie Brugman

Project Contact

/mailto:hennie.brugman@meertens.knaw.nl/

Creator Contact

/mailto:hennie.brugman@meertens.knaw.nl/

Documentation

Help

Multi Tier Annotation Search

Source code

https://github.com/meertensinstituut/mtas

Original source

http://portal.clarin.nl/node/4233

Publications

http://www.nederlab.nl/wp/?page_id=12

Resource

CMDI File Link

License

GNU GPL

Inventory Scope

local

INPOLDER

root — Mon, 08 Apr 2019 08:38:48 +0000

Title

INPOLDER: Integrated Parser and Lemmatizer Dutch in Retrospect

Description

INPOLDER (Integrated Parser and Lemmatizer of Dutch in Retrospect) provides a tool that assigns morphological tagging, lemmatization, and syntactic parsing for historical Dutch texts. It is built on the Adelheid tool (tagging and lemmatization) and Collins-Bikel statistical Parser. As an essential part of the Dutch cultural heritage, it is of vital importance that the Dutch historical record be made accessible for research into a wide range of historical and linguistic research questions. In the transition from the Middle Ages to the modern era, the Netherlands developed from speaking a diverse group of dialects (Hollandic, Brabantic, Flemish, North-eastern, Limburgian) to a country with a standard language, and there is good reason to believe that this process was an extremely dynamic one. Systematic research into these processes affecting syntax, phonology, morphology and spelling cannot be done without access to lemmatized, tagged and parsed corpora of historical Dutch. In recent years, a tagger-lemmatizer has been developed by Hans van Halteren (Adelheid, also available in the CLARIN infrastructure). INPOLDER complements these enrichment tool with a parser for historical Dutch. The INPOLDER parser is trained using a subset of the corpus of fourteenth-century texts (Corpus van Reenen/Mulder CRM, van Reenen and Mulder, 1993; Rem, 2003) and a subset of the Drenthe corpus (DC). CRM consists of 2700 charters from 345 places of origin. The corpus was designed as representative for the local language use of Middle Dutch and to be suitable for all types of linguistic research.

Project

INPOLDER

CLARIN National Project

CLARIN-NL

CLARIN centre

Meertens/HuC

Research domain

Linguistics

Linguistic Subject

historical linguistics

Syntax

Tool task

Gebruikershandleiding en demonstratiescenario bij de INPOLDER web applicatie

constituency-based parsing

morphosyntactic tagging

lemmatisation

Country

Netherlands

Tool Type

web application

Research Phase

Enriching Data

Tool status

published

Input format

text/xml

text/plain

Input Language

Dutch

Access Contact

Gertjan Postma

Marc Kemps-Snijders

Project Contact

Marc Kemps-Snijders

Creator Contact

Prof. Dr. Ans van Kemenade

Documentation

Source code

not specified

Original source

http://portal.clarin.nl/node/1927

Resource

CMDI File Link

License

unknown

Inventory Scope

local

GrETEL 4.0

root — Mon, 08 Apr 2019 08:38:48 +0000

Title

GrETEL Search Engine for Querying Syntactic Constructions in Treebanks

Description

GrETEL Version 4 extends the functionality of GrETEL in a number of ways: (1) it allows uploading one's own corpus, either a parsed corpus in LASSY DTD format, or a text corpus in a variety of formats (currently supported: plain text, CHAT; FoLiA and TEI to follow). An uploaded unparsed corpus will be automatically parsed by Alpino and made available in the search interface. (2) It also enables upload of metadata that are contained in the data, e.g. in accordance with the PaQu metadata format (3) Checking of XPath expressions and suggestions for selecting valid attributes and values have been added, as have the use of macros (a la PaQu) (4) filtering on the basis of metadata has been made possible (5) full analysis of the metadta in combination with nodes and their properties from the syntactic structures is available, allowing a user to select, group, sort, and filter data and metadata in combination, through a pivot table with draggable attributes and node properties. Sums , counts, heatmaps and other graphical visualisation enable rapid analysis of the data. GrETEL 4 is being developed by Utrecht University (UiL-OTS and Digital Humanities Lab) in the context of the AnnCor and CLARIAH-CORE projects. It based on GrETEL 3 developed by KU Leuven. GrETEL is a query engine in which linguists can use a natural language example as a starting point for searching a treebank with limited knowledge about tree representations and formal query languages. Instead of a formal search instruction, it takes a natural language example as input. This provides a convenient way for novice and non-technical users to use treebanks with a limited knowledge of the underlying syntax and formal query languages. By allowing linguists to search for constructions similar to the example they provide, it aims to bridge the gap between descriptive-theoretical and computational linguistics. The example-based query procedure consists of six steps. In the first step the user enters an example of the construction he/she is interested in. In the second step the example sentence is automatically parsed with the Alpino parser, resulting in a parse tree. In the third step the example is returned in the form of a matrix, in which the user specifies which aspects of this example are essential for the construction under investigation. Based on the information indicated in the matrix, a query tree is cut out of the initial parse tree and automatically converted to an XPath query. By clicking the advanced options button, the XPath query is made visible, and can be edited if desired. This query will be used for the actual treebank search. In the fourth step the user can select which treebank needs to be queried. The fifth step provides an overview of the search instruction, i.e. the input sentence, the selected treebank(s), the query tree and the (adapted) XPath query. In the sixth step the query is executed on the selected corpus. The matching constructions are presented to the user as a list of sentences, which can be downloaded. The user can also click on the sentences in order to visualize the results as syntax trees. In addition to the example-based search functionality, users can also query the treebanks using an XPath expression. This query is then processed in the same way as the automatically generated query in the example-based approach.

Project

AnnCor

CLARIAH-CORE

CLARIN National Project

CLARIAH-CORE

CLARIN centre

none yet

Research domain

Linguistics

Linguistic Subject

Syntax

Morpho-syntax

Computational Linguistics

Tool task

corpus searching

corpus exploration

querying

parsing

grammatical relation assignment

analysis

Country

Netherlands

Tool Type

web application

Research Phase