TLK/GFF Tools v1.1.0-pre1
TLK<->XML conversion tools, GFF processors, and more
Tab stop: 4

Copyright (C) 2004-2005 Guido Imperiale
This code is released under the terms of the GNU GPL.
Please refer to COPYING.txt for more info.

http://crusaderky.altervista.org


IMPORTANT NOTE
--------------
tlktools 1.1.0-pre1 is DISCONTINUED; the new functions (those inside the <gff> tag) are only partially implemented. If you're looking for something usable, use tlktools-1.0.3 instead.

There are lots of TODOs, mostly in gff.c and xmlread.c.
If you've got a good knowledge of libxml2, you could finish them.


Note for Linux users
--------------------
libxml2 is required to run this software.
If you're using Gnome v2.x, then you already have it installed; otherwise, you can download it from http://www.xmlsoft.org/


Basic TLK Usage
---------------

tlktools command.xml
Read a complex XML command (See below)

tlk2xml infile.tlk [outfile.xml]
Read a TLK file and write it to an XML file; use standard output if none is provided.

xml2tlk infile.xml outfile.tlk [langcode] [m|f]
Read an XML file and write it to a TLK file.
langcode and sex default to "en" and "m" if none is provided.


The command/data files can be any XML files. Valid tags are the following:

<!--comment-->
	any type of comment, eventually on more than one line.


<tlk></tlk>
	Generic tag, used to open and close the file as well as  set "paragraphs"
	Attributes:
		offset="#"	(optional; default="0")
					Any positive or negative number. From now on, all IDs read will be summed to it.
					This stacks with any previous "offset" attributes of tags that haven't been closed yet.
		lang="xx"	(optional; default="en")
					Language to use as default in the following entries
					Possible values are: en|fr|de|it|sp|po|ko|ct|cs|jp
		sex="m|f"	(optional; default="m")
					Sex to use as default in the following entries
		quiet="0|1"	(optional; default="0")
					Do not print a warning when overwriting existing entries with a different content


<xmlread>file.xml</xmlread>
	Load an XML file. It may contain commands and/or entries, as well as other includes.
	Note that the path refers to the directory where the program is running and not the directory of the XML file.
	Attributes:
		offset="#"	(optional; default="0")
		lang="xx"	(optional; default="en")
		sex="m|f"	(optional; default="m")
		quiet="0|1"	(optional; default="0")


<tlkread>file.tlk</tlkread>
	Load a TLK file.
	Attributes:
		sex="m|f"	(optional; default="m")
			the sex of the TLK file
		offset="#"	(optional; default="0")
		quiet="0|1"	(optional; default="0")


<xmlwrite>file.xml</xmlwrite>
	Write the current talk table to an XML file
	Attributes:
		from="#" (optional)
		         do not write everything, just entries since StrRef # (included)
		to="#"   (optional)
		         do not write everything, just entries until StrRef # (included)

<tlkwrite>file.tlk</tlkwrite>
	Write the current talk table to a TLK file.
	Note that only one sex and language will be written.
	Attributes:
		lang="xx"	(optional; default="en")
		sex="m|f"	(optional; default="m")
	Notes:
		* If an entry exists for only one sex, that one will be used
		* If the entry does not exist for the given language at all, the "en" entry will be used


<entry id="#">Hello World!</entry>
	A TLK entry, eventually on more than one line. Note that, in the text, any '&' will have to be replaced with '&amp;'
		id="#"		(required; decimal only (hexadecimal values, exponentials and sums are not accepted))
					Any number between 0 and 0x01FFFFFF, pointing the StrRef of the entry. Note that the custom table flag is stripped off from IDs, so 0x01000001 is identical to 1.
		lang="xx"	(optional; default="en")
		sex="m|f"	(optional; default="m")
		quiet="0|1"	(optional; default="0")


Please take a look at the sample XML file called sample.xml.



Advanced TLK usage
------------------
Looking for differences between TLKs:

There are two ways to check for differences between TLK versions:
The first, brutish one is to convert both the TLKs to XML and then launch 'diff -u oldtlk.xml newtlk.xml' (it's a UNIX tool. If you're using Windows, you may get it along with a cool UNIX shell with MinGW/MSYS at http://www.crusaderky.altervista.org/downloads.php#KY-MinGW).
The problem is that it will show ALL differences, while you're probably only interested in what you've already modified.

So you may use TLKTools for a more refined solution.


Simple version checking of modified lines:

<tlk>
	<tlkread "sample.tlk">
	<entry id="1" quiet="0">This is the unmodified entry</entry>
	<entry id="1" quiet="1">This is the new entry</entry>
</tlk>

This way, if sample.tlk contains anything different than "This is the unmodified entry" at line 1, a warning will be issued containing the line number and you'll have the chance to modify your patch.


Version checking of translated lines:

<tlk quiet="0">
	<tlkread "sample.tlk">
	<entry id="1" lang="en">This is the unmodified entry</entry>
	<entry id="1" lang="it">Questa  la riga non modificata</entry>
</tlk>


Version checking of translated AND modified lines:

<tlk quiet="0">
	<tlkread "sample.tlk">
	<entry id="1" lang="en"          >This is the unmodified entry</entry>    <!-- Error checking -->
	<entry id="1" lang="en" quiet="1">This is the new entry</entry>           <!-- Modification -->
	<entry id="1" lang="it"          >Questa  la nuova linea</entry>         <!-- Translation of the modified line -->
</tlk>


The <gff> tag
-------------
Using the <gff> tag, you may work on GFF files that have been converted to XML using gff2xml. Basically, you can do two operations:
1)Extract custom LocStrings to a TLK
2)Find, replace and swap instances of several objects, which may be (to name some) TLK entries, tiles, body parts, scripts, etc. This is intended to convert maps and other contents between different HAK packs. For example, you'll need this if you're importing an area file from module A to module B, and you have to move all instances of "human male head" from #45 to #72.

To use this, you need the tools gff2xml and xml2gff from NWN2XML. The newest version can be found at http://sourceforge.net/projects/leto/

IMPORTANT NOTE: most of the operations here are potentially highly destructive; for example, if a <swap> operation somehow doesn't terminate, or no TLK/XML is written after a LocString extraction, you'll end up with broken material. So KEEP BACKUPS and TEST the modified material before deleting them! I will NOT be held responsible for any damage dealt by this software.

A <gff> entry first contains a list of rules, and then a list of GFF files to parse.


<gff></gff>
	Even if it's a bit counter-intuitive, the <gff> tag cannot go outside the tlk tag.
	Attributes:
		extract="0|1"	(optional; default=0)
						When parsing GFF files, extract custom LocStrings to the TLK database. Note that YOU'LL HAVE TO
						SAVE the result using <xmlwrite> or <tlkwrite> afterwards. If no offset is specified, start writing
						at entry 1, else start at offset+0. Already occupied entries will NOT be overwritten.
		offset="#"		Begin writing the extracted LocStrings at the given offset (cumulative with previous ones).
		
<find/>
	Find a given element and print a note to stdout. Do not alter it.
	Attributes:
		Label_Name="#"	(optional; default="Label")	Match the given property
		Label="#"		(mandatory) Find elements with this Label (or custom property)
		Property="#"	(optional; default="Value") 
						change the given property. Use "StringRef" for LocStrings, "Value" for everything else.
		Appearance_Type="#"
		Gender="#"
		Phenotype="#"
		Tileset="#"
		Baseitem="#"	(optional) Consider only if processing the given creature appearance/gender/phenotype, 
						the given tileset or baseitem ID (from baseitems.2da)
		id="#"			(mandatory) Value to look for. Generally it's treated as a case-sensitive string; however, if
						property="StringRef", it is NOT treated as a string; you must use the "d" prefix to distinguish between
						entries from the default TLK and custom ones. so "d1" means 0x00000001, while "1" means 0x01000001.


<replace/>
	Replace any occourences of an element.
	Attributes:
		Label_Name="#"
		Label="#"
		Property="#"
		Appearance_Type="#"
		Gender="#"
		Phenotype="#"
		Tileset="#"
		Baseitem="#"
		id="#"
		from="#"		Alternate name for "id".
		to="#"			(mandatory) The replacement target.

<swap/>
	Swap any occourences of "from" and "to".
	Attributes:
		Label_Name="#"
		Label="#"
		Property="#"
		Appearance_Type="#"
		Gender="#"
		Phenotype="#"
		Tileset="#"
		Baseitem="#"
		id="#"
		from="#"
		to="#"

<ftlk/>
    Abbreviation for <find    Label_Name="Type" Label="LocString" Property="StringRef"/>

<rtlk/>
    Abbreviation for <replace Label_Name="Type" Label="LocString" Property="StringRef"/>

<stlk/>
    Abbreviation for <swap    Label_Name="Type" Label="LocString" Property="StringRef"/>


<parse>file.gff.xml</parse>
	Parse the file using the previously defined rules. It has to be a GFF file converted to XML by GFF2XML.

<parse_list>list.txt</parse_list>
	Parse a text list of files to parse, one file per row, generated for example by the "find" command.


Some examples:

Look for all dialogs that use the script obsolete.nss
<find Label="Script" id="obsolete"/>

Transform all Balors to Badgers
<replace Label="Appearance_Type" from="38" to="8"/>

Replace "Thin Human Male" head #48 with #32
<replace Label="Appearance_Head" Appearance_Type="6" Gender="0" Phenotype="0" from="48" to="32"/>

Swap tile #288 with #274 in the Forest tileset
<swap Label="Tile" Tileset="ttf" from="288" to="274"/>

Move all TLK references in LocStrings from 0x01000000+54 (custom TLK) to 726 (standard TLK)
<rtlk from="54" to="d726"/>


Compilation
-----------
To build this program from sources, you need
* Any POSIX shell with gcc and make (tested on Linux and CygWin)
* The libxml2 library and development files, that can be downloaded from http://www.xmlsoft.org/

libxml2 is available for Linux, Solaris, MacOS-X and Cygwin for Windows. No MinGW/MSYS support is available at the moment.
You could try building this using MS Visual C++, but no Makefile is provided and you'll have to do everything by yourself.

1)position yourself in the src/ directory
2)launch 'make all'
3)install the executable files (tlktools[.exe], tlk2xml[.exe], xml2tlk[.exe]) wherever you please.



Known Issues
------------

* Not tested on BigEndian systems (Macintosh) but it SHOULD work.
* Not tested on 64-bits CPUs (i.e. Athlon64) but it SHOULD work. If it doesn't, compile it with the -m32 gcc flag.
* Not tested with non-european character sets.
* No sounds support.
* Paths are relative to the present working directory (pwd) and not to the XML files.



Changelog
---------

1.1.0
-----
* Added the <gff> subsection
* xmlwrite: skip female version if it's identical to the male one
* Added "from" and "to" parameters to xmlwrite
* Fixed (but not tested!) Athlon64 bug

1.0.3
-----
* Now, even if not quiet, you won't get a warning if overwriting the same string (excellent for version control!)
* Added the "Advanced Usage" section to the README file
* Improved sample.xml

1.0.2
-----
* Fixed crash in the windows version introduced in 1.0.1

1.0.1
-----
* Fixed charset handling bug
* Fixed non-escaped & for sex="f"
