|Monday, August 6th, 2007|
|Tracker will be in Gutsy and on by default
As mentioned on Digg : Finally Ubuntu will add fast indexed search Tracker to Gutsy
This means Gutsy tribe 4 will ship tracker in main and be on by default :)
This has been our number one goal (as ubuntu is my fav distro) and its real great for tracker to get the recognition it deserves. It should also mean that tracker will be thoroughly integrated into ubuntu with nautilus and deskbar support out of the box (in the future we will add Rhythmbox and Epiphany support too so they can benefit from tracker - just need XESAM to be implemented first)
Its also not just about search as tracker has one big advantage over nearly all other indexers - its a metadata store that can deliver a semantic desktop with tagging, shared metadata, big performance improvements and a host of features that will help deliver a next generation desktop (see my guadec slides for more info here
I just can't wait to integrate it more...
|Tuesday, July 24th, 2007|
|Tracker 0.6 released
Just released a new version of tracker which is bursting with tons of cool new functionality
As you can see from the above screenshot, we have used a sidebar to separate service categories (we flirted with tabs before but they were not HIG compliant). You also asked for a better GUI and more services indexed and we have delivered big time on this. All major services are now indexed (files, emails, chat logs). We are also leading the way to a semantic desktop with tagging everywhere.
* Evolution Emails (mbox/pop/imap) now indexed
* Gaim/Pidgin chat logs now indexed
* Applications now indexed
* New files indexed - all spreadsheets, csv files (these require gnumeric) and oo draw
* Vastly improved and new search UI using a sidebar for categories and a new metadata tile
* Tagging support in the UI
* New tracker-preferences UI for easy setting of prefs
* Tons of bug fixes - all major bugs squished!
* libtracker-gtk widget toolkit
* Now uses XDG directories
* Optimised triple store
* Handles multiple triple stores (files, emails and user data are all separate dbs for speed and safety)
* User defined and indexed data are stored separately
* New IDF based ranking algorithm
* XMP metadata support added
* Made use of async calls in UI to prevent UI freezing
* Added 60 second delay before indexing
* Added support for crawling directories without watching overhead
(very useful for indexing source code without causing slowdowns when
compiling - an often requested feature from devs who find tracker slows down their compiling when watching)
Its been a while coming but we wanted to fix all the major bugs and the new services (emails especially) introduced more bugs which took a fair bit of time to nail.
Apologies to the bugsquad who have been waiting a few months for some fixes to stop dupes coming through. I guess we will need to release far more frequently in future rather than take months to fix too much in one go.
Big thank you to the many and awesome contributors to tracker who have made this release really rock - Im confident its all been worth the wait!
PS Also dont forget our new config UI :
|Monday, May 14th, 2007|
|Patents - the last refuge of a scoundrel
Groklaw has some interesting comments on Microsoft's new patent extortion scheme
and its well worth a read.
Its great to know all this with GPL3 just around the corner because it gives us a chance to thwart their plans.
And if there is one big change I would love to see in GPL3 it would be to finally bury the patent fiasco for good. By that I dont just mean overturning or preventing the Novell/MS patent deal but actually provide full indemnification via OIN
for *all* users and distributors of GPL3 software.
OIN would need a substantial cash injection from the big boys as well as a patent war chest to provide the necessary indemnification and then all thats needed is a clause in GPL3 to automatically confer this protection.
Micorsoft's game has always been to frighten off the competition and now we have a chance to prevent them cynically leaching and extorting unearned revenue from OSS end users. So please lets stop them while we still have the time to modify GPL3.
|Thursday, May 3rd, 2007|
|Tracker will have a nice API...
It will be called XESAM
In response to the plea of Emmanuele
(and a big thank you for sorting out the gtk filechooser search stuff), the existing APIs of tracker, beagle, strigi and other indexing engines will become less relevant as XESAM takes shape and provides a common dbus interface for search and metadata. And you can bet there will be a nice gobject client lib for XESAM which should suffice for languages which dont have nifty dbus bindings.
I dont want to waste time developing new APIs for something thats going to become deprecated in the near future (Id rather spend the time implementing XESAM support in tracker and beagle has a SoC for doing likewise)
Of course if there is anything not covered by XESAM in tracker then we can look at creating nicer gobject based APIs for those too.
We also really need to encourage devs to use XESAM as soon as its finished and available as it will save a lot of duplication of code and needless squabbles over the "one true indexing system".
|Monday, April 30th, 2007|
|Dell to preinstall Ubuntu?
Seems the rumour that Dell will choose Ubuntu (which has been going around ever since the CEO had it on his laptop) now appears to have some truth in it
Anyway if its true then let me be the first to congratulate Canonical/Ubuntu. Fesity is simply the best there is especially as it excels on lesser hardware and is a breeze to install more powerful stuff like beryl for those with suitable machines. Its fast, slick and most things "just work".
Im sure this will go a long way to fixing the famous bug #1
|Thursday, March 8th, 2007|
|Vote Tory for more Open Source usage in Government
At long last a political party
has finally seen the light and the massive savings to made from going the open source route.
On another note, with both Blair and Brown close friends of Bill Gates I am a bit cynical you will ever see the labour party embrace this so the Tories look like being onto a winner.
I was going to vote Tory anyhow as its much more likely there will be a full inquiry into the Iraq war if they are elected (and hopefully amass enough evidence to put the scandalous Blair in prison on war crimes charges)
|Sunday, March 4th, 2007|
|Document Browser for Gnome
There seems to be no limit to the number of great ideas that tracker could provide.
Heres one from Marko for a GNOME Document Browser.
I like the idea a lot and it could fit in nicely with the referencer
project that was blogged on pgo recently here
The powerful combinaton of extensible metadata database and indexer in tracker should make all this real easy to implement.
It would rock if we could have a whole lot of these : a MediaTracker, a DocumentTracker, a PhotoTracker etc. Im thinking of adding widgets to our new libtracker-gtk widget library to facilitate development of these kinds of apps (like a general purpose search result grid or tree that has basic file management stuff built-in which could easily be combined with our existing tagging widgets so no need to reinvent the wheel each time).
|Friday, March 2nd, 2007|
|All that glitters is not gold
The BBC has a nice article
about the pain of installing Vista - incredibly it sounds a lot like Linux on the desktop a few years ago (IE having to dig deep into the system to get things working). Its also rife with incompatibility and most machines will need a lot of upgrading to get those pretty windows.
In short - its going to cost you a lot of time and money.
Ironically modern distributions like Fedora, Ubuntu and Suse are a lot more ready for the desktop than that pile of junk and do the business with a lot less.
Now if only we could turn this into our advantage...
|Tuesday, February 13th, 2007|
|Nautilus Metadata Tile
integration is taking place in Nautilus courtesy of Neil J. Patel
(who incidentally is the author of the rocktastic awn
Hes been busy creating the Nautilus metadata tile
that uses tracker to pull out the metadata and allow tagging too.
click on screenshot to see full size:
Its pretty nice and I will probably reuse it in tracker-search-tool.
Rock on Neil!
|Tracker at your service
Tracker now has a much nicer and improved UI to go hand in hand with new services being indexed like Evolution emails
The tab view provides a nice breakdown by service type of the search results complete with a hit count in each tab (services without hits are not shown)
Also any searched email can be double clicked and the email will be opened in evolution and shown in its own window.
Expect a release some time this week as I iron out the few remaining bugs/issues.
I'm sure a few distro's will start considering shipping tracker by default in the coming months now that the all important email indexing is just about done and especially as we tackle more and more services which will come thick and fast now.
|Tuesday, February 6th, 2007|
|A Metadata Enabled GNOME
The potential for tracker
integration in GNOME goes way beyond the likes of indexers and desktop search thanks to its extensible metadata database (which makes tracker truly unique, useful and really innovative).
An extensible metadata rich service can provide a host of benefits and some of these are nicely summed up by John Stowers post "A-Metadata-Enabled-GNOME"
who is also working on *making things happen* rather than shouting from the rooftops.
Its also another reason why tracker is actually bloody useful (considerably more than its competitors which are simply dedicated indexers) and can help eliminate speed/memory problems like the multitude of .desktop files that have to be loaded in to the GNOME menus. A rant
that is all too common these days but can be easily solved
with trackerTracker Update
I have got Evolution email indexing (with summary files and correct URIs so indexed emails can be opened in Evolution) working nicely and will soon upload and release a version with full email support. We need to update our UI too to handle emails better so expect a revamped tracker-search-tool too in the next few days
Also Saleem Abdulrasool has submitted a nice patch to provide a capplet app to configure tracker and its really nice!
The next release of tracker is going to be absolutley fab!
|Friday, January 26th, 2007|
|Tracker 0.5.4 - Speed Daemon
A new much improved version of tracker
is now available for your enjoyment.
New Features :
* Indexing at *ludicrous* speed - massively optimised indexing so its now 10x faster than previous version. Indexing speeds are now around 100 text files per second (which is about the maximum possible considering the I/O time to read 100 files from a hard drive). You wont find many indexers faster than this!
* Index while you work - now scheduled even better so it should never slow the system down whilst allowing users to work without being affected by or even noticing it.
Tracker will not hammer your system nor consume unreasonable amounts of memory and is virtually unobtrusive in daily use.
Also provides additional --throttle command line parameter to customise throttling even further so no need to endure noisy fans or hot laptops while indexing. The throttle param can take a value between 0 (fastest) and 20 (slowest) allowing for fine tuning (not that you will need it in most cases as the default value 0 should not slow your system down)
* New verbosity param to control how much data is shown on screen and in log
* Fixed mplayer backend which is now used if gstreamer backend returns no metadata
* Improved reaping of all spawned apps - no more zombies!
* Improved mime and text file detection
* Added improved deskbar-handler
* Got tracker to run nicely on FUSE based filesystems
Tracker is rapidly evolving into the best application in its class with virtually unbeatable indexing speed, excellent memory efficiency, rock solid stability, high quality snowball stemmers, blazing fast search and of course our semantic web RDF-like metadata database (which no other indexer currently provides) which should give us the edge over competing platforms like OS/X and Vista.
The only thing missing is more service support and the good news is that's now on the way. Our email code is 99% there but it needs a wee bit more testing and our gui updated for it so expect it within the next two weeks.
Also on the agenda is additional built in support for Chat logs and desktop files (applications). On top of that we are planning to use desktop files to specify third party extractors, filters and external services to be indexed. So keep your eyes peeled for a big increase in service support in tracker!
|Tuesday, December 5th, 2006|
|MS and Open XML wins by hook or by crook!
I dont want to get into habit of bashing certain companies but I cant help expressing my further concern that this so called *interoperability* between OpenOffice and MS Office Open XML is not what it seems. Indeed I'm pretty certain its one of those shabby secret deals Novel's CEO made with MS in return for all that money.
Firstly, I have to ask why now?
Is Open XML such a widely used format that it requires urgent support in OpenOffice?
Is it anywhere close to being ubiquitous or even remotely widespread to justify its inclusion?
If the answer to the above is yes then of course OpenOffice has to support it just like it has to with any de facto standard but if thats not the case then one has to suspect ulterior motives.
So is the real reason simply that MS wanted to destroy the case that Massachusetts Information Technology Division made for adopting ODF in the first place? (namely that all publications should use a freely available implementation of ODF to prevent any kind of tax being levied in order to read them)
With OpenOffice acquiring the ability to read Open XMl, has MS just pulled the rug from under ODF?
It seems like MS is now getting its money's worth as all their previous attempts to thwart ODF support in Massachusetts have failed.
And now it gets to beat Massachusetts with their own argument - if Open XML is freely available in other free packages why then should anyone switch from Ms Office or use ODF?
And to top it off, Ms does not need to support ODF in any shape or form in its products to win here so MS can simply leverage its monopoly power to effectively dictate what formats are used instead.
Its an argument that Massachusetts will have a hard time answering and its just typical - just when thought you MS was on the ropes and that finally its monopoly is going to get smashed, MS finds a way by hook or by crook to reassert its dominance and eliminate the competition.
I do feel cynical about this and I really hope I have got it all wrong but I really cant see how we can *ever* beat MS if they can simply buy out our allies and use them for their own purposes against us and our interests.
|Tuesday, November 21st, 2006|
I have just released a new version of tracker
Tarball can be found here
Edgy debs in usual place (see readme for how to install)
This version fixes a few bugbears with metadata extraction. We now have timeouts for all extractors (fulltext and metadata) so tracker should never hang while extracting and we have also set an upper memory limit in extractors to prevent runaway memory usage which is usually caused by a large binary file being mistaken for a video file by XDGMime (which consequently gets mapped entirely into memory while hunting for non-existant metadata). Our use of setrlimit here should prevent big memory spikes from occurring as a result of this. This should help make tracker an industrial strength indexer with unprecedented reliability and robustness while extracting.
Also we have polished our GNOME GUI tracker-search-tool which now correctly word wraps search snippets (why oh why is there not a setting in GtkCellRendererText to wrap text to column size automatically?) and we have also given it a speed boost by fetching search snippets asynchronously and by using the mime type from tracker instead of the slower sniffed gnome-vfs ones.
There has also been improvements to our search snippet generator which now uses a copied g_utf8_strcasestr from GtkSourceView (why isn't this extremely useful function in GLib?) as well as a few fixes to prevent crashes and improve highlighting of matched terms (especially stemmed words as shown above - "technology" matching against "technologies").
Also big thanks to Luca Ferreti for sorting out the l10n/i18n support in t-s-t and tracker's command line tools and Deji Akingunola who helped get tracker in Fedora extras and produced desktop files for us. Laurent Aguerreche has also done good work in improving our filter support and sorting out the debian builds.
See here for full details of all changes
And before I forget, we now have a page to list all our supported file formats complete with what additional apps/packages that tracker needs to index them. (yeah I know I need to update the other tracker webpages!)
|Sunday, November 12th, 2006|
|If a deal is too good to be true then it often is
And that is my opinion on the MS/Novell deal.
I have really big issues with it because we dont know all the facts and Novell has not come clean with the entire contents of the deal (most of the released stuff and FAQs are smoke and mirrors IMO). If there is an NDA preventing disclosure then you have to ask why? My conclusion is because there's sensitive and extremely controversial stuff in there that Novell does not want you to know about.
Of course I could be wrong but its pushing the bounds of credibility to believe that Novell has not made some dodgy commitments in return for all that money and I have to assume its money that the CEO of Novell was most interested in at the expense of everything else (as Novell is under great pressure to make a return on their linux investment). Whilst its true that Novell has contributed a lot to Linux and therefore deserves some trust, that cannot be said of the new CEO of Novell as far as I can tell.
I find it difficult to believe that MS would give so much especially when it appears that Novell did not seem to have any leverage going into the deal (in stark contrast to the Sun/MS deal where Sun obviously has MS by the balls with legal action)
I dont really want to speculate on what shabby secret deals the CEO of Novell has made but my gut feelings are that Novell will act as an MS proxy in some of its dealings (EG if MS wanted to amend ODF then Novell may be contractually bound to help them). This of course is complete speculation but then what do you expect if you have a confidential agreement with our greatest enemy who's back stabbing exploits are legendary?
There is a way out of course - Novell can send back the money and tell MS the deal's off.
And as I said previously, FSF are actively looking at plugging the hole that MS has exploited in GPL3 so its pretty clear how this deal will backfire if they dont.
I joined GNOME to get away from MS and I really dont want them here in any shape or form so please Novell do the right thing
|Saturday, November 11th, 2006|
|Actions speak louder than words
There been a lot of criticism lately on planet gnome about the Novell/microsoft pact.
If you really disapprove of whats going on and what to send a message that these kind of agreements are unacceptable then its time to take action rather than mince words.
I would suggest the following:
1) Make sure GPL 3 removes the loophole that Novell are exploiting.
2) Help the FSF get GPL3 out the of door ASAP
3) When GPL3 is ready, make sure that all future improvements to any software you own, maintain or hack on are exclusively protected by GPL3+
The end result : Novell will only be able to ship old versions of your GPL2 compliant software for the duration of their Micorsoft agreement
Their Suse distro will therefore not be able to ship a lot of the latest and greatest software for 5 years (unless they can revoke the agreement) and should deter others from entering into similar deals.
|Wednesday, November 8th, 2006|
I released a new version of tracker yesterday
You can find the tarball here
Debs for Edgy can be found here
and for Debian Unstable here
There are also efforts
underway to get this into Fedora Extras
Due to the threading problems in some versions of SQLite that I mentioned earlier, we have now included by default a static linked threadsafe version of sqlite. It can still be built without this using the --enable-external-sqlite compile option.
Its important that anyone using 0.5.0 upgrade right away to avoid potential problems from this.
An updated screenshot of the slightly tidied up tracker-search-tool is below:
|Friday, November 3rd, 2006|
I really enjoy reading RedHat's Truth Happens
page as it provides a comprehensive list of links to the controversial Novell <-> Microsoft pact.
(and for the cynical its more like a "shit happens" page!)
Nevertheless it looks like RedHat is furious about this pact though Im not sure "innovation tax" really means anything if there is a net flow of royalties to Novell from Microsoft.
What worries me though is the new influence over a lot of important open source stuff that Microsoft might have acquired.
With the pact being confidential in nature it will be hard to separate fact from FUD here as others have speculated.
So I guess some clarity over what Microsoft is getting out of all this would be nice, anybody?
|Thursday, November 2nd, 2006|
I have now published a roadmap for tracker's development over the next few months:
November 2006: Implement an advanced optimised metadata store similar to rdf triple store complete with metadata relationships.
December 2006: Implement Live query with HAL support (this is needed for RhythmBox integration and allows us to pick up music files automatically when plugging in a usb drive).
January 2007: Finish off outstanding services to be indexed including emails, chat logs, applications and possibly vCard-based contacts
February 2007: More integration polish such as a new GtkFileChooser, enhanced search GUI etc
March 2007+ : Subject to it being accepted in GNOME 2.18, we will then further integrate it into GNOME with Epiphany, Yelp, GEdit and any others if they are willing...
|Wednesday, November 1st, 2006|
|Sqlite and threads
With Tracker 0.5.0 out of the barn door, there has been a major issue that has reared its ugly head
With tracker now using Sqlite as its database of choice, one factor that has been a pain is that sqlite does not have threadsafe mode enabled by default in most distro's including Edgy (you can check sqlite3.pc and see if the libs section has pthread in it - if not then there is no guaranteed thread protection).
Strangely, I was not getting any corruption or problems without the thread protection as it appears that threads respects each other's file locks although this of course cannot be relied upon as file locking is an unreliable mess!
Several users on tracker's mailing list were not so fortunate and complained of crashes and content not being indexed. One of those, however, compiled sqlite with --enable-threadsafe and all their errors and problems instantly disappeared so a solution is definitely needed here even though the problem is with current distribution of sqlite and not tracker.
Two solutions spring to mind :
1) protect sqlite calls in tracker with mutexes
2) inline sqlite as a private statically linked library with threadsafe directive enabled
(1) turned out to have really poor performance as its not really possible to make it fine grained enough to avoid significant lock contention so (2) turned out to be the optimal solution especially as there was no perceptible loss of performance.
Tracker CVS now has sqlite embedded and statically linked in. Thanks to Sqlite actually being pretty light as its name suggest, the static linking only added about 400kb to the tracker daemon (static linking also improves performance too as well as removing another dependency!)
I will be releasing a 0.5.1 version soonish with this...