Fielddb

An offline/online field database which adapts to its user's terminology and I-Language.

View project onGitHub

NPM version Build Status Dependency Status

FieldDB is a free, modular, open source project developed collectively by field linguists and software developers to make an expandable user-friendly app which can be used to collect, search and share your data, both online and offline. It is fundamentally an app written in 100% Javascript which runs entirely client side, backed by a NoSQL database (we are currently using CouchDB and its offline browser wrapper PouchDB alpha). It has a number of webservices which it connects to in order to allow users to perform tasks which require the internet/cloud (ie, syncing data between devices and users, sharing data publicly, running CPU intensive processes to analyze/extract/search audio/video/text). While the app was designed for "field linguists" it can be used by anyone collecting text/audio/video data or collecting highly structured data where the fields on each data point require encryption or customization from user to user, and where the schema of the data is expected to evolve over the course of data collection while in the "field."

FieldDB was officially launched in Spanish on August 1st 2012 in Patzun, Guatemala as iCampo an app for fieldlinguists. Since then more than 500 users at 50+ universities that we know of have started using and giving us feedback about the app. You can find more about the "non-technical" side of the project at LingSync.org or find "technical" publications about the project on here on this page. You can also watch screencasts which demo how parts of the app & infrastructure work by searching on YouTube.

Visit LingSync.org, the site for users...

Publications

  • Dunham, J., A. Bale and J. Coon Feb 28 2015. "LingSync: web-based software for language documentation." ICLDC 4. abstract
  • Dunham, J. Jan 8 2015. "LingSync." LSA 2015. abstract, slides1, slides2, slides3, slides4, video1, video2, video3, video4, video5
  • Bale, A., J. Coon and M. Wagner. Jan 8 2015. "LingSync and ProsodyLab-Aligner: Tools for Linguistic Fieldwork and Experimentation", LSA Workshop 2015. abstract
  • Little, C-R. Nov 21 2014. "LingSync Fieldwork Software Demo." Cornell Language Documentation Lab.
  • Dunham, J. Nov 10 2014. "LingSync Tutorial." LRC I’d Like to Know More About ... Workshop. UCalgary. blurb
  • Dunham, J and E. McClay. September 12 2014. "LingSync Tutorial." UBC. slides
  • Chiodo, G., E. Chkhikvadze. August 20 2014. Recognizing Speech on Android: ანდროიდის ქართველი მომხმარებლებისთვის სიტყვის ამოცნობის სისტემის შექმნა. Android Montreal. slides, video, installer
  • Dunham, J, G. Chiodo, and J. Horner. June 26 2014 "LingSync & the Online Linguistic Database: New models for the collection and management of data for language communities, linguists and language learners." ComputEl: The use of computational methods in the study of endangered languages, 52nd Annual Meeting of the Association for Computational Linguistics. paper, video, slides
  • Dunham, J and E. McClay. May 8 2014. "LingSync Tutorial." Presentation at Exploring the Interfaces (ETI) 3. McGill University, Montreal, QC. video, slides
  • Bielig, L. Nov 22, 2013. "Linguistic Corpus-building for Low-Resource Languages: an Inuktitut case study." ms McGill, University. tex
  • McClay, E., H. Noguchi, E. Olson, C. Little, A. Bale, J. Coon & G. Chiodo. Oct 3, 2013. “Using technology to bridge gaps between speakers, learners, and linguists.” Electronic poster and demo at the annual Foundation for Endangered Languages Conference, Ottawa, ON. slides
  • McClay, E., H. Noguchi, E. Olson, C. Little, A. Bale, J. Coon & G. Chiodo. June 3, 2013. “Using technology to bridge gaps between speakers, learners, and linguists.” Poster presented at the Special Session on Indigenous Languages at the annual Canadian Linguistic Association conference, Victoria, BC. abstract
  • Bielig, L., J. Horner, C. Little, T. Skinner. May 28, 2013. "Plugging into LingSync." Presentation at Computational Field Workshop, McGill University, Montreal, QC.
  • Noguchi, H., G. McCulloch, G. Chiodo. May 27, 2013. "Why LingSync." Presentation at Computational Field Workshop, McGill University, Montreal, QC. slides.
  • Manyakina, Y. Feb 20, 2013. "LingSync: an on/offline database app for field linguistics." Brown Bag Presentation at Stony Brook University.
  • Deering, T., G. Chiodo. Oct 9, 2012. "Taking Javascript into the field: a case study." Presentation at JSMontreal. slides, video.
  • McClay, E. "Language Learning App: Learn Mi'gmaq prototype". Oct 5, 2012. whitepaper, prototype.
  • Cathcart, M.E., G. Chiodo, T. Deering, Y. Manyakina, G. McCulloch, H. Noguchi. Aug 1, 2012. "LingSync: A Free Tool For Creating And Maintaining A Shared
 Database For Communities, Linguists, And Language Learners.
" Proceedings of FAMLi 2: Formal Approaches to Mayan Linguistics. MITWPL 74. prepress paper, publisher
  • Cathcart, M.E., G. Chiodo, T. Deering, Y. Manyakina, G. McCulloch, H. Noguchi. Aug 1, 2012. "iCampo: Un aplicación de colección de datos lingüísticos." MITWPL 74. Presentation at Corpus Approaches to Mayan Languages Workshop, Patzun, Guatemala. demostración
  • LingSync WhitePaper whitepaper

Documentation

User Documentation

Code Documentation

We use the Agile/Scrum software development method. Scrum (10 minute) meetings are held almost every day, Monday-Friday at 12:30pm EST (Montreal time) in this Google Hangout. You are welcome to join us. All features must be documented in the feature/issue tracker, and can be found simply by running a search in the feature tracker or by browsing in progress and completed Milestones.

Behavior Driven Development

All features must also be documented in a Jasmine Specification ('spec') which is a type of documentation which compiles, and can say whether or not the code meets the specification. Specs are useful both when the code is written, but also over time as a project grows in complexity. A spec is a way of detecting a regression, a loss of features as new features break old ones. Specs and feature trackers are a common low-effort way of documenting a decentralized software project.

Models and Data Structures (JSDocs)

If you would like to see the classes/models in FieldDB and how they can interact and be used, you can browse the API documentation which was autogenerated using a tool called 'jsdoc' (documentation generator for JavaScript). Class docs can help software developers/interns who are already know the project understand how to use the source files.
Class Documentation (src) (html)

IEEE documentation

If you are a student of Computer Science/Software Engeering and you would like to help keep the IEEE documentation up to date please contact us on the issue tracker.

White Paper

The white paper was written in April 2012 and discusses how the project will meet E-MELD and DataOne best practices in data management, as well as some of the modules which were planned and estimates of how much time it would take to build them. The White Paper is perhaps interesting to users, and maybe help users situate the project as compared to similar software.
White Paper (.tex), (pdf)

Software Design Description (SDD)

A Software Design Description IEEE 1016-2009 is a representation of a software design to be used for communicating design information to its stakeholders. It generally contains diagrams which help visualize the software system and maybe most useful for SOEN or CompSci interns, or academics who wish to learn more about they can think like a software engineer (both academic and industry).
Software Design Description (.tex), (pdf)

Interns and Development Team

Funding

We would like to thank SSHRC Connection Grant (#611-2012-0001) and SSHRC Standard Research Grant (#410-2011-2401) which advocates open-source approaches to knowledge mobilization and partially funded the students who have doubled as fieldwork research assistants and interns on the project. We would like to thank numerous other granting agencies which have funded the RAs and TAs who have also contributed to the project as interns. If you have a student/RA who you would like to customize the project for your needs, contact us at support @ lingsync . org

License

This project is released under the Apache 2.0 license, which is an very non-restrictive open source license which basically says you can adapt the code to any use you see fit.

How to Contribute Code

  • Signup for a GitHub account (GitHub is free for OpenSource)
  • Click on the "Fork" button to create your own copy.
  • Leave us a note in our issue tracker to tell us a bit about the bug/feature you want to work on.
  • You can follow the 4 GitHub Help Tutorials to install and use Git on your computer.
  • You can watch the videos on YouTube dev playlist and/or in the Developer's Blog to find out how the codebase works, and to find where is the code that you want to edit. You might also like the user tutorial screencasts to see how the app is supposed to behave. Feel free to ask us questions in our issue tracker, we're friendly and welcome Open Source newbies.
  • Edit the code on your computer, commit it referencing the issue #xx you created ($ git commit -m "fixes #xxxx i changed blah blah...") and push to your origin ($ git push origin master).
  • If you know how, you should also try to run the tests $ npm install and $ grunt test it should say something like Finished in 10.388 seconds 732 tests, 2308 assertions, 0 failures, 0 skipped screen shot 2015-02-20 at 11 53 50 am Then you can also run the entire build $ grunt travis to make sure your changes dont affect other parts of the app. If any of these parts errors, ask us for help in the issue tracker.
  • Click on the "Pull Request" button, and leave us a note about what you changed. We will look at your changes and help you bring them into the project!
  • Feel the glow of contributing to OpenSource :)

Related repositories

These are the webservices which the FieldDB clients use, and which make up the complete FieldDB suite. If you fork the project, you might also be intersted in forking these repositories and adapting them to your needs. We created two scripts to simplify the process of downloading and building the FieldDB dependancies into one directory.

Core webservices:

Optional webservices

  • Audio webservice (for hosting audio files and running processes such as the ProsodyLab's Aligner)
  • Lexicon webservice (for search functionality, and glosser functionality if you are a linguist)

Client apps/libraries:

You might also be interested in the code for some of the client apps which use FieldDB api/corpora.