<chapter id="importing-exporting">
<title>Importing and Exporting Data</title>

<para>
&appname; is able to import and export a wide variety of data files, as well as search various Internet sites for information.
</para>

<sect1 id="internet-search">
<title>Importing from the Internet</title>
<para>
&appname; is able to search various Internet sites using the <interface>Internet Search Dialog</interface>. Entries may be directly imported and added to your current collection. The various providers are configured via the <link linkend="data-sources-options">Data Sources Options</link>.
</para>

<para>
Searches may use different criteria: <guilabel>Title</guilabel>, <guilabel>Person</guilabel>, <guilabel>ISBN</guilabel>, <guilabel>UPC/EAN</guilabel>, <guilabel>LCCN</guilabel>, or <guilabel>Keyword</guilabel>. Not all criteria are available for certain data sources. <acronym>ISBN</acronym> and <acronym>LCCN</acronym> values are for books only, while <acronym>UPC</acronym> or <acronym>EAN</acronym> values can apply to any type of item.</para>

<para>Once a search is initiated, the <guibutton>Search</guibutton> button becomes <guibutton>Stop</guibutton> which will end a search. As results are found, they are added to the list directly under the search box, where selecting an item will show the result without adding it to the collection. Clicking the <guibutton>Add Entry</guibutton> button will add all the selected items to your collection. If the data source has more results than were initially requested, the <guibutton>Get More Results</guibutton> button becomes active. Clicking <guibutton>Clear</guibutton> will remove all the current results and reset the search.
</para>

<screenshot>
<screeninfo>The &appname; <interface>Internet Search Dialog</interface></screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="fetch-dialog.png"/></imageobject>
<textobject><phrase>The &appname; <interface>Internet Search Dialog</interface></phrase></textobject>
</mediaobject>
</screenshot>

<para>
Only entries that match the current collection type will be found. The <guilabel>Description</guilabel> column provides additional information about the entry, in order to differentiate between videos in different formats, or books in different bindings, for example. Once an entry is successfully added to the collection, a checkmark is added to the first column in the list.
</para>

<para>
Multiple entries can be added at once by using the standard &kde; method for multiple selection, which usually involves holding the &Shift; or &Ctrl; key when clicking on an item.
</para>

<para>
To facilitate the use of barcode scanners, searches can include multiple ISBN/UPC values. Selecting the <guilabel>Multiple ISBN/UPC search</guilabel> check box will disable the search box and enable the <guibutton>Edit ISBN/UPC values...</guibutton> button, which will open a multi-line text entry box. Each ISBN should be entered on a line by itself. After closing the box, each ISBN will be validated for correct formatting. The ISBN validation is able to convert 13-digit EAN values, as well as full <acronym>UPC</acronym> codes, to the proper formatting. The ISBN list may also be read from a text file.
</para>

</sect1>

<sect1 id="importing">
<title>Importing Data</title>

<para>
&appname; offers three different actions when importing data. <guilabel>Replace current collection</guilabel> will close the current collection, and create a new one with the data from the imported file. <guilabel>Append to current collection</guilabel> tells &appname; to add all the entries in the imported collection to the current one, and to add any fields which don't currently exist. The <guilabel>Merge collection</guilabel> action is the same as appending, except that each imported entry is compared to the current ones, and any identical entries are skipped. &appname; attempts to identify matching entries which are not completely identical by comparing significant fields and will then merge the entries. For example, music collections compare the artist and album, and the tracks would be merged for matching entries. The <link linkend="importing-audio-files">audio file importer</link> is able to correctly build track lists by merging entries.
</para>

<sect2 id="alternate-software">
<title>Importing Data From Other Software</title>

<para>&appname; can import data directly from a variety of other collection management programs, including
  <application><ulink url="https://gitlab.com/GCstar/GCstar">GCstar</ulink></application>,
  <application><ulink url="https://onmyshelf.app">OnMyShelf</ulink></application>,
  <application><ulink url="https://www.datacrow.net/">Data Crow</ulink></application>,
  <application><ulink url="https://github.com/mvz/alexandria-book-collection-manager">Alexandria</ulink></application>,
  <application><ulink url="https://www.delicious-monster.com">Delicious Library</ulink></application>,
  <application><ulink url="https://www.collectorz.com">Collectorz</ulink></application>,
  <application><ulink url="http://www.antp.be/software/moviecatalog/">Ant Movie Catalog</ulink></application>,
  <application><ulink url="https://en.wikipedia.org/wiki/Referencer">Referencer</ulink></application>,
  and <application><ulink url="https://launchpad.net/griffith">Griffith</ulink></application>.</para>

</sect2>

<sect2 id="import-data-files">
<title>Importing Other Data Formats</title>
<para>&appname; can import data from a variety of other file formats, including <acronym>CSV</acronym>, bibtex, audio discs and files, <acronym>MODS</acronym>, &PDF;, and <acronym>RIS</acronym>.</para>

<sect3 id="importing-tellico">
<title>Importing &appname; Data</title>

<para>
Other &appname; data files may be imported directly. Replacing the current collection by importing a &appname; file is the same thing as just opening the file itself. The value of importing &appname; data is primarily for appending or merging two collections together.
</para>
</sect3>

<sect3 id="importing-csv">
<title>Importing CSV Data</title>

<para>
Comma-separated values (<acronym>CSV</acronym>) are a common way of importing and exporting tabular data. Each field value is separated by a comma, with one entry per line. The field titles may be included in the first line. The CSV importer is not limited to using a comma as the separator. Any character or string may be used.
</para>

<screenshot>
<screeninfo>The CSV Import Dialog</screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="csv-dialog.png"/></imageobject>
<textobject><phrase>The CSV Import Dialog</phrase></textobject>
</mediaobject>
</screenshot>

<para>
First, select the type of collection that you are importing. If you are appending or merging to your open collection, the type is limited to your current collection type. If the first line of the CSV file contains the field titles, click the check box and the importer will automatically compare the titles against the fields in the current collection. If a field title matches, the header for that column changes to show that the column has been assigned to that field. If the file uses a delimiter other than a comma, be sure to change that option accordingly.
</para>

<para>
In order for &appname; to properly import the file, it must know which field corresponds to each column. <emphasis>If the column only has a number in the header, the data in that column will not be imported.</emphasis> You should assign fields to each column by selecting a column, either by clicking in it or changing the column number itself, then selecting the field to assign from the drop down box and clicking the <guibutton>Assign Field</guibutton> button. If you need to add a new field to the collection, the last item in the drop down box opens the  <link linkend="fields-dialog"><interface>Collection Fields Dialog</interface></link>.
</para>

<para>
For compactness, only the first five lines of the imported CSV file are shown in the dialog. However, all the lines in the file will be imported.
</para>

</sect3>

<sect3 id="importing-audio">
<title>Importing Audio &CD; Data</title>

<para>
&appname; is able to use the <ulink url="http://gnudb.org">freedb.org</ulink> service to lookup
information about a &CD;, including the track list. Depending on your distribution,
settings for access to the service may be set in the &kde; &systemsettings;. The &CD; artist, title, genre,
year, and track listing are all added.
</para>

<para>
In addition, if the disc contains &CD;-Text, that information is read and added to the imported entry.
</para>
</sect3>

<sect3 id="importing-audio-files">
<title>Importing Audio File Metadata</title>

<para>
&appname; is able to scan a folder and read the tags for common audio file formats, such as <literal role="extension">mp3</literal> and <literal role="extension">ogg</literal>. The songs are entered in a music collection, where each entry is an album. If the song files contain the track number, the song name is inserted in the correct spot in the track list. The artist and genre information is also added to the entry. If the song tags contain comments, they are appended to the comments field in the entry, preceded by the file name.
</para>

<para>
In addition, if a folder contains a <filename>.directory</filename> file and the folder name matches an album title, the <literal>Icon</literal> entry in the desktop file is used as the cover image for the album.
</para>

<para>
The audio file metadata importer can recursively scan a folder to find all audio files in any subfolder, though symbolic links are not followed. &appname; uses the <ulink url="https://taglib.org">TagLib library</ulink> for reading the audio file metadata, and so can import data from any file type that TagLib understands.
</para>

</sect3>

<sect3 id="importing-bibtex">
<title>Importing Bibtex Data</title>

<para>
<ulink url="http://en.wikipedia.org/wiki/Bibtex">Bibtex</ulink> is a bibliography format used with the LaTeX document preparation system. Various type of bibliographic references may be included in the file. &appname; imports bibtex files as a <link linkend="bibliography">Bibliographic collection</link>.
</para>

<para>
If the bibtex importer encounters fields in the file which are not in the default bibliography collection, they are added as <link linkend="line">Simple Text</link> fields, with two exceptions. If the field value contains more than 100 characters, it becomes a <link linkend="para">Paragraph</link> field. If the field value appears to contain a &URL; or a file reference, then a <link linkend="url">&URL;</link> field is created. &appname; uses an internal copy of the <ulink url="http://www.gerg.ca/software/btOOL/">btparse library</ulink> for parsing the bibtex files.
</para>

<para>
<ulink url="http://bibtexml.sf.net">Bibtexml</ulink> is an &XML; representation of bibtex data, and the data from the imported bibtexml file is treated in the same way as bibtex data would be.
</para>
</sect3>

<sect3 id="importing-mods">
<title>Importing MODS Data</title>

<para>
<acronym><ulink url="http://www.loc.gov/standards/mods/">MODS</ulink></acronym> is a format for representing various types of media collections. Currently, only books are imported by &appname;, as a <link linkend="bibliography">Bibliographic collection</link>.
</para>

</sect3>

<sect3 id="importing-pdf">
<title>Importing &PDF; Data</title>
<para>
If &appname; was compiled with <emphasis>exempi</emphasis> or <emphasis>poppler</emphasis> support, metadata from &PDF; files can be imported. Metadata may include title, author, and date information, as well as bibliographic identifiers which are then used to update other information.
</para>
</sect3>

<sect3 id="importing-ris">
<title>Importing RIS Data</title>
<para>
The <ulink url="https://en.wikipedia.org/wiki/RIS_(file_format)"><acronym>RIS</acronym> format</ulink> is a bibliographic file format used by <application>EndNote</application>, <application>Reference Manager</application>, and others. &appname; imports RIS files as a <link linkend="bibliography">Bibliographic collection</link>.
</para>
</sect3>

</sect2>

<sect2 id="import-online-collection">
<title>Importing Online Collections</title>
<para>&appname; can connect to and import from websites that manage personal collections.</para>

<sect3 id="importing-boardgamegeek">
<title>Importing BoardGameGeek Collection</title>
<para>
<ulink url="https://boardgamegeek.com">BoardGameGeek</ulink> is an online board gaming resource and community. &appname; can import the board games in a user's collection, as long as the collection is set to be publicly accessible. The imported collection may be limited to those items marked as being owned.
</para>
</sect3>

<sect3 id="importing-goodreads">
<title>Importing Goodreads Collection</title>
<para>
<ulink url="https://www.goodreads.com">Goodreads</ulink> is an online social network for readers to track book collections. &appname; can import the list of books in a user's collection, given either the user name or user ID, as long as the collection is set to be publicly accessible.
</para>
</sect3>

<sect3 id="importing-librarything">
<title>Importing LibraryThing Collection</title>
<para>
<ulink url="https://www.librarything.com">LibraryThing</ulink> is an online service to help people catalog their books easily. &appname; can import the list of books in a user's collection, <ulink url="https://www.librarything.com/export.php?export_type=json">exported in JSON format</ulink>.
</para>
</sect3>

<sect3 id="importing-discogs">
<title>Importing Discogs Collection</title>
<para>
<ulink url="https://www.discogs.com">Discogs</ulink> is an online database and marketplace of music releases. &appname; can import the list of music releases in a user's collection, given  the user name, as long as the collection is set to be publicly accessible.
</para>
</sect3>

</sect2>

<sect2 id="importing-file-listing">
<title>Importing File Listings</title>

<para>
&tellico; can scan folders to import file information, primarily to create a <link linkend="file-catalog">File Catalog</link> collection. The folder may be searched recursively, to add all files found within. This importer is most useful for backup listings and media cataloging, such as &CD; or <acronym>DVD</acronym> listings. In addition, image previews of the file contents may be generated. The file previews are same as those shown in the &kde; file manager.
</para>

<para>
Folders with ebook files and video files may also be imported as book and video collections. File metadata can be read, and for video files, any existing <literal role="extension">nfo</literal> files are used to complete the video entry information.
</para>

</sect2>

<sect2 id="importing-xslt">
<title>Importing &XML; Data via XSLT</title>

<para>
Any &XML; file may be imported into &appname; provided an &XSL; stylesheet is available to convert the file to &appname; format. &appname; automatically loads the stylesheet and performs the &xslt; processing needed to load the file.
</para>
</sect2>

</sect1>

<sect1 id="drag-n-drop">
<title>Drag and Drop</title>

<para>Dragging data files to the main &appname; window and dropping them will import the files, just as if the <link linkend="importing">import command</link> was made from the menus. Drag and drop works for the following file formats: Tellico, Bibtex, RIS, &PDF;, and EPub. Importing multiple files at once is also supported.</para>

<para>So, for example, if you want to catalog several <link linkend="importing-pdf">&PDF; files</link>, select them in the file manager and drag them to the &appname; window. &appname; will import as much metadata from the files as it can, and then fetch additional information from various configured Internet sources.</para>

</sect1>

<sect1 id="exporting">
<title>Exporting Data</title>

<para>
When exporting the data in the collection, the entry values may be exported as entered, or with the <link linkend="field-formatting">automatic formatting</link> provided by &appname;. Additionally, the export may be limited to the currently selected entries of the collection as well, where the <link linkend="status-bar">statusbar</link> shows the number of selected entries.
</para>

<para>
Exported text files, such as Bibtex or CSV, may use the Unicode (UTF-8) character encoding, or the current locale of the operating system.
</para>

<screenshot>
<screeninfo>General Export Options</screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="export-options.png"/></imageobject>
<textobject><phrase>General Export Options</phrase></textobject>
</mediaobject>
</screenshot>

<sect2 id="exporting-xml">
<title>Exporting &XML;</title>

<para>
The file format for &appname; data is a zipped &XML; file. Exporting to &XML; merely creates the &XML; file without zipping it. Images may be included in the &XML; file as base64-encoded data in an image element, but doing so can create very large text files.
</para>
</sect2>

<sect2 id="exporting-zip">
<title>Exporting Zip</title>

<para>
The standard file format for &appname; is a zipped file, containing the &XML; collection file, and optionally, all the images referenced in the collection. If the images are being stored in the application folder instead, exporting to a Zip file will create a self-contained data file, which includes all the images in the collection.
</para>
</sect2>

<sect2 id="exporting-html">
<title>Exporting &HTML;</title>

<para>
The &HTML; export uses the <filename>tellico2html.xsl</filename> stylesheet. Images are exported to a folder with the same name as the exported &HTML; file with <emphasis><filename>_files</filename></emphasis> appended.
</para>

<para>
The default format is similar to the printed output, and allows various options for modifying the &HTML;. Field headers may be printed at the top of each column, but unfortunately, &kde; does not yet allow the table headers to be repeated on each page. The entries may be grouped as in the <interface>Group View</interface>, as well.
</para>

<para>
Additionally, individual files may be created for each entry in the collection, with links created in the top-level &HTML; file. The entry files will be created in the same folder as the images. The entry &HTML; files will use the current stylesheet template, as shown in the <interface>Entry View</interface>.
</para>

<screenshot>
<screeninfo>&HTML; Export Options</screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="export-html.png"/></imageobject>
<textobject><phrase>&HTML; Export Options</phrase></textobject>
</mediaobject>
</screenshot>
</sect2>

<sect2 id="exporting-csv">
<title>Exporting CSV</title>

<para>
Comma-separated values (CSV) are a common way of importing and exporting tabular data. Each field value is separated by a comma, with one entry per line. The field titles may be included as headers in the first line. Any character or string other than a comma may also be used to delimit the fields.
</para>

<screenshot>
<screeninfo>CSV Export Options</screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="export-csv.png"/></imageobject>
<textobject><phrase>CSV Export Options</phrase></textobject>
</mediaobject>
</screenshot>
</sect2>

<sect2 id="exporting-alexandria">
<title>Exporting Alexandria</title>

<para>
<application><ulink url="https://github.com/mvz/alexandria-book-collection-manager">Alexandria</ulink></application> is a book collection manager for the <ulink url="http://www.gnome.org">GNOME</ulink> desktop environment. &appname; is able to export a limited subset of book collection fields to the default Alexandria data location.
</para>

</sect2>

<sect2 id="exporting-onix">
<title>Exporting ONIX</title>

<para>
<ulink url="http://www.editeur.org/onix.html">ONIX</ulink> is an &XML; format for representing and communicating book industry product information, primarily for book vendors. &appname; can export book collections using a small subset of ONIX.
</para>

</sect2>

<sect2 id="exporting-bibtex">
<title>Exporting Bibtex</title>

<para>
When exporting to <ulink url="http://en.wikipedia.org/wiki/Bibtex">Bibtex</ulink>, the field values may be escaped with braces or quotation marks. If any string macros
are used in the collection, they may optionally be exported as macros or expanded. For &URL; fields, &appname;
may enclose the field values with the <literal>\url{...}</literal> tag. Finally, entries with no citation key may be skipped rather than have &appname; auto-generate the key.
</para>

<screenshot>
<screeninfo>Bibtex Export Options</screeninfo>
<mediaobject>
<imageobject><imagedata format="PNG" fileref="export-bibtex.png"/></imageobject>
<textobject><phrase>Bibtex Export Options</phrase></textobject>
</mediaobject>
</screenshot>
</sect2>

<sect2 id="exporting-gcstar">
<title>Exporting GCstar</title>

<para>
<application><ulink url="http://www.gcstar.org">GCstar</ulink></application> is another movie collection manager. &appname; is able to export most collection types to a GCstar data file.
</para>

</sect2>

<sect2 id="exporting-xslt">
<title>Exporting &XML; via &xslt;</title>

<para>
Finally, &appname; is able to process its internal &XML; representation of the collection data through an external &XSL; stylesheet before exporting. This type of export may be useful for generating text reports or other file types.
</para>
</sect2>

</sect1>

<sect1 id="citations">
<title>Working With Citations</title>
<para>
When working with a <link linkend="bibliography">bibliography</link>, citations for the currently selected entries
may be generated and used in various other applications. A citation in bibtex format can be copied to the clipboard,
and then pasted in a latex file. Bibtex citations can also be pushed to an external application such as <application>LyX</application> or <application>Kile</application> using the so-called <emphasis><link linkend="hidden-bibtex-options">lyxpipe</link></emphasis>.
</para>

</sect1>

</chapter>
