Traži
Exact matches only
Search in title
Search in content
Search in comments
Search in excerpt
Search in posts
Search in pages
Search in groups
Search in users
Search in forums
Filter by Custom Post Type
Filter by Categories
Uncategorized
Resources and tools
Resource type
Tools
Lexicons
Corpora
Language
Serbian
Croatian
News
Data collection instruments
Linguistic subfield
Syntax
Sociolinguistics
Second language acquisition
Pragmatics
Neurolinguistics
Language
Serbian
English
Croatian
Instrument type
Non-experimental instruments
Experimental instruments

Regional Linguistic Data Initiative

23 Feb

Posted at 11:18h in Croatian, Resources and tools, Serbian, Tools

A Python module comprising of a tokeniser, a part-of-speech/MSD tagger, a lemmatiser, a dependency parser, and a named entity recognizer for most South Slavic languages. For Croatian and Serbian there are models for processing standard and Internet non-standard texts. The estimated accuracy of morphosyntactic tagging for this tool is ~94%, while for lemmatisation the accuracy is ~99%. Dependency parsing has an labeled attachment score of ~0.9, while named entity recognition achieves a micro-F1 of ~0.9. Author Nikola Ljubešić Publications The experiments yielding this pipeline...

23 Feb

Posted at 11:04h in Corpora, Resources and tools, Serbian

The SentiComments.SR dataset includes the following three corpora: The main SentiComments.SR corpus, consisting of 3490 movie-related comments The movie verification corpus, consisting of 464 movie-related comments The book verification corpus, consisting of 173 book-related comments The main SentiComments.SR corpus was constructed out of the comments written by visitors on the kakavfilm.com movie review website in Serbian. The movie verification corpus comments were sourced from two other Serbian movie review websites - gledajme.rs and happynovisad.com. The book verification corpus comments were also...

10 Dec

Posted at 20:42h in Corpora, Resources and tools, Serbian

The Serbian Semantic Textual Similarity News Corpus (STS.news.sr) consists of 1192 pairs of sentences in Serbian, or around 64 thousand tokens, gathered from news sources on the web and written in the Serbian Latin script. Each sentence pair was manually annotated with fine-grained semantic similarity scores on the 0-5 scale. The final scores were obtained by averaging the individual scores of five annotators. The sentence pairs in this dataset were taken from the Serbian Paraphrase Corpus (paraphrase.sr). The annotation methodology generally...

10 Dec

Posted at 02:24h in Corpora, Resources and tools, Serbian

The Serbian Paraphrase Corpus (paraphrase.sr) consists of 1194 pairs of sentences gathered from news sources on the web. Each sentence pair was manually annotated with a binary similarity score that indicates whether the sentences in the pair are semantically similar enough to be considered close paraphrases. The corpus contains 553 sentence pairs deemed to be semantically equivalent (46.31% of the total number), and 641 semantically diverse pairs (53.69% of the total number). Author Vuk Batanović Availability The corpus and its documentation can be found...

11 Aug

Posted at 12:26h in Corpora, Resources and tools, Serbian

ReLDI-NormTagNER-sr 2.1 is a manually annotated corpus of Serbian tweets. It is meant as a gold-standard training and testing dataset for tokenisation, sentence segmentation, word normalisation, morphosyntactic tagging, lemmatisation, and named entity recognition of non-standard Serbian. Each tweet is also annotated for its automatically assigned standardness levels (T = technical standardness, L = linguistic standardness). Authors Nikola Ljubešić, Tomaž Erjavec, Vuk Batanović, Maja Miličević, Tanja Samardžić Availability For local use, a full-text version of the corpus can be downloaded from the CLARIN.SI repository. Publication The corpus...

11 Aug

Posted at 12:16h in Corpora, Croatian, Resources and tools

ReLDI-NormTagNER-hr 2.1 is a manually annotated corpus of Croatian tweets. It is meant as a gold-standard training and testing dataset for tokenisation, sentence segmentation, word normalisation, morphosyntactic tagging, lemmatisation, and named entity recognition of non-standard Croatian. Each tweet is also annotated for its automatically assigned standardness levels (T = technical standardness, L = linguistic standardness). Authors Nikola Ljubešić, Tomaž Erjavec, Vuk Batanović, Maja Miličević, Tanja Samardžić Availability For local use, a full-text version of the corpus can be downloaded from the CLARIN.SI repository. Publication The corpus...

24 Jun

Posted at 11:40h in Corpora, Serbian

The Serbian Movie Review Dataset collection consists of three movie review datasets in Serbian which were constructed for the task of sentiment analysis: Collected movie reviews in Serbian (ISLRN 252-457-966-231-5) - an imbalanced collection of 4725 movie reviews in Serbian. SerbMR-2C - The Serbian Movie Review Dataset (2 Classes) (ISLRN 016-049-192-514-1) - a two-class balanced dataset that contains 1682 movie reviews (841 positive and 841 negative). SerbMR-3C - The Serbian Movie Review Dataset (3 Classes) (ISLRN 229-533-271-984-0) - a three-class...

24 Jun

Posted at 10:58h in Croatian, Serbian, Tools

This package is a Java reimplementation of four previously published stemming algorithms for Serbian and Croatian: The greedy and the optimal subsumption-based stemmer for Serbian, by Vlado Kešelj and Danko Šipka A refinement of the greedy subsumption-based stemmer, by Nikola Milošević A "Simple stemmer for Croatian v0.1", by Nikola Ljubešić and Ivan Pandžić All the stemmers expect the input text to be formatted in UTF-8. Their outputs are also UTF-8 encoded. Author Vuk Batanović Availability The package and a more extensive documentation can be downloaded...

31 May

Posted at 12:04h in Croatian, Serbian, Tools

This tool is considered a legacy tool as the NLP pipeline achieves better results on the same task, but is not available as a web service yet. A tool for automatic lemmatisation (returning the base or dictionary form of an inflected word). The tool looks up the hrLex/srLex lexicons and uses a predictive model for lemmatising OOVs (out of vocabulary words) which was trained on available corpora and lexicons. Author Nikola Ljubešić Availability The lemmatiser is freely available in three forms: For local use, the...

31 May

Posted at 11:59h in Corpora, Serbian

SETimes.SR is a reference training corpus of Serbian texts collected from the SETimes parallel news corpus. It contains 163 documents divided into 3891 sentences, or 86 726 tokens. The corpus is manually annotated on the following levels: Token, sentence, and document segmentation Morphosyntax Lemmas Dependency syntax Named entities The set of morphosyntactic tags used in the corpus follows the revised MULTEXT-East V5 tagset for Bosnian, Croatian and Serbian, available here. Dependency syntax is annotated according to the Universal Dependency v2 specification (UDv2). Named entity annotations...

31 May

Posted at 11:58h in Corpora, Croatian

hr500k is a reference training corpus of Croatian that consists of 900 documents divided into 24 794 sentences, or 506 457 tokens. It is an extension of previous training corpora for Croatian, such as SETimes.HR and SETimes.HR+. The corpus is manually annotated on the following levels: Token, sentence, and document segmentation Morphosyntax Lemmas Dependency syntax Semantic roles Named entities The entire corpus was annotated with regard to morphosyntax and lemmas. The set of morphosyntactic tags used in the corpus follows the revised...

02 May

Posted at 20:16h in Language, Lexicons, Resource type, Resources and tools, Serbian

srLex is an inflectional lexicon of Serbian. The size of the lexicon is 169,328 lemmas, or 6,905,941 surface forms. Each entry in the lexicon consists of a (wordform, lemma, MSD, MSD features, UPOS, morphological features, absolute frequency, in-million frequency) 8-tuple. The frequencies were estimated on the Serbian web corpus srWaC. The set of morphosyntactic tags used in the lexicon follows the MULTEXT-East V6 tagset for Serbo-Croatian macro-language, available here. Authors Nikola Ljubešić Availability For local use, srLex can be downloaded as a raw text file here. srLex can...

02 May

Posted at 20:14h in Croatian, Language, Lexicons, Resource type, Resources and tools

hrLex is an inflectional lexicon of Croatian. The size of the lexicon is 164,206 lemmas, or 6,427,709 4,970,520 surface forms. Each entry in the lexicon consists of a (word form, lemma, MSD, MSD features, UPOS, morphological features, absolute frequency, in-million frequency) 8-tuple. The frequencies were estimated on the Croatian web corpus hrWaC. The set of morphosyntactic tags used in the lexicon follows the MULTEXT-East V6 tagset for Serbo-Croatian macro-language, available here. Authors Nikola Ljubešić Availability For local use, hrLex can be downloaded as a raw text file here. hrLex...

02 May

Posted at 12:07h in Corpora, Serbian

srWaC is a web corpus collected from the .rs top-level domain. The 1.1 version of the corpus contains 555 million tokens. The corpus is automatically annotated on the diacritic restoration, morphosyntax and lemma layers. The dependency syntax layer will be added in version 1.2. The set of morphosyntactic tags used in the corpus follows the revised MULTEXT-East V5 tagset for Croatian and Serbian, available here. Authors Nikola Ljubešić, Filip Klubička Availability For local use, a full-text version of srWaC can be downloaded here. srWaC can also...

02 May

Posted at 12:03h in Corpora, Croatian, Language, Resource type, Resources and tools

hrWaC is a web corpus collected from the .hr top-level domain. The 2.1 version of the corpus contains 1.4 billion tokens. The corpus is automatically annotated on the diacritic restoration, morphosyntax and lemma layers. The dependency syntax layer will be added in version 2.2. The set of morphosyntactic tags used in the corpus follows the revised MULTEXT-East V5 tagset for Croatian and Serbian, available here. Authors Nikola Ljubešić, Filip Klubička Availability For local use, a full-text version of hrWaC can be downloaded here. hrWaC can also...

02 May

Posted at 12:00h in Croatian, Serbian, Tools

A tool for automatic diacritic restoration on text with potentially missing diacritics (e.g. it turns kuca into kuća if necessary). Reported accuracy of the tool: 99.5% on standard language and 99.2% on non-standard language. Authors Nikola Ljubešić, Tomaž Erjavec, Darja Fišer Availability The tool is freely available in two forms: The code and models of the tool can be downloaded from this GitHub repository. Our web service can be accessed from of our Python library, which can also be downloaded from the CLARIN.SI GitHub...

02 May

Posted at 11:49h in Croatian, Serbian, Tools

This tool is considered a legacy tool as the NLP pipeline achieves better results on the same task, but is not available as a web service yet. A tool for automatic tokenisation (dividing text into words and sentences). It was engineered through iterative runs on representative datasets and features modes for both standard and non-standard language. Authors Nikola Ljubešić, Tomaž Erjavec Availability The tokeniser is freely available in three forms: For local use, the tokeniser can be downloaded from this GitHub repository. The tokeniser can...

02 May

Posted at 11:43h in Croatian, Language, Resource type, Resources and tools, Serbian, Tools

This tool is considered a legacy tool as the NLP pipeline achieves better results on the same task, but is not available as a web service yet. A tool for automatic annotation on the morphosyntactic level. It is capable of tagging both Croatian and Serbian as models for both languages are present in the tool. The tagger is based on the CRF algorithm trained on a 500,000-token Croatian training corpus and the hrLex/srLex lexicons for each respective language. The set of morphosyntactic tags used...