The GStreamer extractor often needs more time than 5 seconds

Martyn Russell authored Apr 21, 2009 and Jürg Billeter committed Apr 24, 2009

* src/tracker-extract/tracker-extract.c: Added debugging so we know
  which modules are loaded on start up and to know which modules are
  used to extract content on a per file basis to help when we have
  multiple extractors handling the same mime types.

* src/tracker-extract/tracker-main.c: Change logging format when
  running stand alone and not writing to a log file and don't log
  everything, only log based on the verbosity setting.

	* src/libtracker-common/tracker-thumbnailer.h:
	* src/libtracker-common/tracker-albumart.h: Added preprocessor
	checks for __LIBTRACKER_COMMON_INSIDE__ which were missing.

	* src/libtracker-common/tracker-albumart.[ch]: 
	* src/tracker-extract/tracker-extract-mp3.c: Fixed coding style
	issues.

	* src/libtracker-common/tracker-albumart.[ch]: 
	* src/tracker-extract/tracker-extract-albumart.c:
	* src/tracker-extract/tracker-main.[ch]: Only create one
	TrackerHal object during the life time of tracker-extract instead
	of for EVERY file which has album art (which is quite a lot). This
	seems to be showing an increase of 2x the speed now for
	tracker-extract. 

	* src/tracker-indexer/tracker-main.c: (main): Added print
	statement like we have in the daemon so we know when the process
	is finished and returning to the cmd line.

	* tests/tracker-extract/tracker-extract-test-utils.c:
	(tracker_test_extract_file_access): Added tracker_main_g...

2009-04-02  Carlos Garnacho  <carlos@imendio.com>

        * src/tracker-extract/tracker-extract-gstreamer.c
        (tracker_extract_gstreamer): Fix a leak and a possible memory
        corruption.
        * src/tracker-extract/tracker-extract-mp3.c (extract_mp3): Do not make
        id3 tags parsers read invalid memory if file size is above
        MAX_FILE_READ.
        * src/tracker-extract/tracker-extract-png.c (extract_png): Free
        correctly the ending info pointer.
        * src/tracker-extract/tracker-extract.c (get_file_metadata): Plug a
        leak, caused when no extractor was found.
        * src/tracker-extract/tracker-main.c: Initialize the quit timeout ID
        to 0, so we don't try to remove nonexistent sources.

svn path=/trunk/; revision=3167

svn path=/trunk/; revision=3156

	* configure.ac:
	* src/tracker-extract/Makefile.am: 
	* src/tracker-extract/tracker-extract-vorbis.c: Added vorbis
	checks and cleaned up vorbis code. This is a backup, we use
	GStreamer normally and this is disabled by default.

	* src/tracker-extract/tracker-extract-abw.c:
	* src/tracker-extract/tracker-extract-jpeg.c:
	* src/tracker-extract/tracker-extract-mp3.c:
	* src/tracker-extract/tracker-extract-png.c:
	* src/tracker-extract/tracker-extract-ps.c: 
	* src/tracker-indexer/modules/evolution-imap.[ch]: Use
	tracker_file_() functions to open/close files with
	posix_fadvise(). 

	* src/tracker-extract/tracker-extract.c: Fixed 2 memory leaks.

svn path=/trunk/; revision=3154

	* data/dbus/tracker-extract.xml:
	* src/tracker-extract/tracker-extract.[ch]:
	* src/tracker-indexer/tracker-indexer.c:
	* src/tracker-indexer/tracker-module-metadata-utils.c: Added
	GetPid API call to extractor. We now call this furst so we know
	what process to kill when the extractor misbehaves. The old
	solution would kill EVERY tracker-extract process on the system
	which (if the user has permissions) is potentially quite bad for
	multiple users using Tracker. This should be quicker than the old
	solution too.

svn path=/trunk/; revision=3120

	* src/tracker-extract/tracker-escape.c: 
	* src/tracker-extract/tracker-extract-gstreamer-playbin.c:
	* src/tracker-extract/tracker-extract-gstreamer.c:
	* src/tracker-extract/tracker-extract-jpeg.c:
	* src/tracker-extract/tracker-extract-mp3.c: 
	* src/tracker-extract/tracker-extract.c:
	* tests/tracker-extract/tracker-extract-test-utils.c: Don't use
	g_strstrip() before calling tracker_escape_metadata() which calls
	g_strstrip() for us.

svn path=/trunk/; revision=3002

svn path=/trunk/; revision=2964

2009-02-24  Carlos Garnacho  <carlos@imendio.com>

        * src/tracker-extract/tracker-extract.c:
        * src/tracker-extract/tracker-main.c: call alarm() prior to processing
        a file, so the extractor dies if it gets stuck.

svn path=/trunk/; revision=2963

	* src/tracker-extract/tracker-extract.c: Check file size is
	non-zero before trying to extract metadata from it. This saves us
	time and warnings in the log.

	* src/tracker-extract/tracker-main.c: Added GOption based command
	line argument handling. Also added -m for mime -f for filename and
	-v to set the verbosity to something other than the default config.

svn path=/trunk/; revision=2910

	* src/tracker-extract/tracker-extract.c: Use the new DBus debug
	function so we don't spam the logs in EVERY log verbosity mode.
	This makes quite an impact on indexing time with verbosity set to 3.

svn path=/trunk/; revision=2904

	* src/tracker-extract/tracker-escape.c: Make sure we check for
	UTF8 validation before sending over dbus.

	* src/tracker-extract/tracker-extract.c: Fixed some warnings
	related to NULL hash tables.

svn path=/trunk/; revision=2897

	* src/tracker-extract/dummy.c:
	* src/tracker-extract/tracker-extract-abw.c:
	* src/tracker-extract/tracker-extract-exif.c:
	* src/tracker-extract/tracker-extract-gstreamer-helix.c:
	* src/tracker-extract/tracker-extract-gstreamer-tagreadbin.c:
	* src/tracker-extract/tracker-extract-gstreamer.c:
	* src/tracker-extract/tracker-extract-html.c:
	* src/tracker-extract/tracker-extract-imagemagick.c:
	* src/tracker-extract/tracker-extract-jpeg.c:
	* src/tracker-extract/tracker-extract-libxine.c:
	* src/tracker-extract/tracker-extract-mp3.c: 
	* src/tracker-extract/tracker-extract-mplayer.c:
	* src/tracker-extract/tracker-extract-msoffice.c:
	* src/tracker-extract/tracker-extract-oasis.c: 
	* src/tracker-extract/tracker-extract-pdf.c: 
	* src/tracker-extract/tracker-extract-playlist.c:
	* src/tracker-extract/tracker-extract-png.c:
	* src/tracker-extract/tracker-extract-ps.c:
	* src/tracker-extract/tracker-extract-tiff.c:
	* src/tracker-extract/tracker-extract-totem.c:
	* src/tracker-extract/tracker-extract-vorbis.c:
	* src/tracker-extract/tracker-extract-xmp.c: 
	* src/tracker-extract/tracker-xmp.c: Renamed 'Extractor' to 'Extract' 

	* data/dbus/Makefile.am:
	* data/dbus/org.freedesktop.Tracker.Extract.service.in:
	* data/dbus/tracker-extract.xml:
	* src/tracker-extract/Makefile.am:
	* src/tracker-extract/tracker-extract.[ch]: New DBus object with
	introspection to request metadata for a file and mime type.

	* src/tracker-indexer/Makefile.am:
	* src/tracker-indexer/tracker-dbus.h:
	* src/tracker-indexer/tracker-module-metadata-utils.c: Call the
	new extracter over DBus.

	* tests/tracker-extract/tracker-extract-jpeg-test.c:
	* tests/tracker-extract/tracker-extract-mp3-test.c:
	* tests/tracker-extract/tracker-extract-png-test.c:
	* tests/tracker-extract/tracker-extract-test-utils.c:
	* tests/tracker-extract/tracker-extract-test-utils.h:
	* tests/tracker-extract/tracker-extract-testsuite-avi.c:
	* tests/tracker-extract/tracker-extract-testsuite-generic.[ch]:
	* tests/tracker-extract/tracker-extract-testsuite-jpeg.c:
	* tests/tracker-extract/tracker-extract-testsuite-mp3.c:
	* tests/tracker-extract/tracker-extract-testsuite-png.c:
	* tests/tracker-extract/tracker-extract-testsuite-tiff.c:
	* tests/tracker-extract/tracker-extract-tiff-test.c: Updated tests
	accordingly.

svn path=/trunk/; revision=2895

svn path=/trunk/; revision=2893

2009-01-30  Philip Van Hoof  <philip@codeminded.be>

	* src/tracker-indexer/tracker-main.c
	* src/trackerd/tracker-main.c
	* src/tracker-extract/tracker-extract.c
	* src/libtracker-common/tracker-thumbnailer.c
	* src/libtracker-common/tracker-thumbnailer.h: No timeout to wait
	for for tracker-extract


svn path=/trunk/; revision=2870

2009-01-14  Philip Van Hoof  <philip@codeminded.be>

        * src/tracker-extract/tracker-extract.c: Fixed too early shutdown of
        the thumbnailer support code in tracker-extract


svn path=/trunk/; revision=2791

2008-12-18  Carlos Garnacho  <carlos@imendio.com>

        * src/tracker-extract/tracker-escape.[ch]: New methods to escape
        metadata properly, so we don't barf on new lines, metadata separators,
        etc...
        * src/tracker-extract/Makefile.am: Add new files.
        * src/tracker-extract/tracker-extract.h: Include header for modules.
        * src/tracker-extract/tracker-extract-*.c:
        * src/tracker-extract/tracker-xmp.c: Escape metadata properly in the
        right places.

svn path=/trunk/; revision=2733

2008-12-17  Carlos Garnacho  <carlos@imendio.com>

        * src/tracker-extract/tracker-extract.c (print_meta_table_data):
        Escapemetadata values so we allow multiline data and other stuff.
        * src/tracker-indexer/tracker-module-metadata-utils.c
        (metadata_utils_get_embedded): And compress it back here.

svn path=/trunk/; revision=2726

svn path=/trunk/; revision=2723

2008-12-17  Philip Van Hoof  <philip@codeminded.be>

	* src/trackerd/tracker-main.c
	* src/tracker-extract/tracker-extract.c: Initialization of the thumbnailer


svn path=/trunk/; revision=2722

svn path=/trunk/; revision=2689

	* src/libtracker-data/tracker-data-metadata.[ch]:
	* src/libtracker-data/tracker-data-query.[ch]:
	* src/tracker-extract/tracker-extract.c:
	* src/tracker-indexer/modules/applications.c:
	* src/tracker-indexer/modules/evolution-imap.c:
	* src/tracker-indexer/modules/evolution-pop.c:
	* src/tracker-indexer/tracker-indexer.c: 
	* src/tracker-indexer/tracker-metadata-utils.c:
	* src/tracker-indexer/tracker-module-file.c: Added more variable
	checking to the tracker_data_metadata_() public API. Now all calls
	to the tracker_data_metadata_() API take a const argument and are
	copied, not inherited. This also fixes a bug where we were double
	freeing a GList.

svn path=/trunk/; revision=2631

	* src/libtracker-common/tracker-type-utils.[ch]: 
	* src/tracker-extract/tracker-extract.[ch]: 
	* src/tracker-extract/tracker-extract-mp3.c: 
	* src/tracker-extract/tracker-extract-gstreamer.c: Revert changes
	that removed "tracker:unknown" when we didn't know the metadata
	value for some fields. Now we have METADATA_UNKNOWN defined in
	tracker-extract.h so we can be consistent everywhere with this.
	Moved tracker_generic_date_to_iso8601() here from
	tracker-extract.c since it is so common.

	* src/tracker-extract/tracker-extract-exif.c: Code clean up and
	remark that this module is not used.

	* src/tracker-extract/tracker-extract-imagemagick.c:
	* src/tracker-extract/tracker-extract-jpeg.c:
	* src/tracker-extract/tracker-extract-ps.c: 
	* src/tracker-extract/tracker-extract-tiff.c: Code clean up and
	used the new date function in tracker-type-utils.

	* src/tracker-extract/tracker-extract-png.c: Fixed a warning and
	used the new date function in tracker-type-utils.

svn path=/trunk/; revision=2532

	* src/tracker-extract/tracker-extract.c: Added g_type_init, as GIO is 
	being used for some operations


svn path=/trunk/; revision=2404

	* src/tracker-extract/Makefile.am: Put all definitions and
	includes before CFLAGS (like everywhere else). Fix linking order.

	* src/tracker-extract/tracker-extract.c: Don't call g_strstrip()
	on NULL values, check the mime type first since it can be NULL.
	Fixes a warning that was being produced. Also added GOption
	support so users can do --help to see how to use the extractor
	from the terminal.

svn path=/trunk/; revision=2348

2008-10-08  Carlos Garnacho  <carlos@imendio.com>

        * src/tracker-extract/tracker-extract.c (print_file_metadata)
        (process_input_cb) (main): Make tracker-extract process inmediately
        and exit if path/mimetype are passed through argv. It will be useful
        for testing purposes.

svn path=/trunk/; revision=2328

svn path=/trunk/; revision=2275

svn path=/trunk/; revision=2274