Managing Devices 12/15/2011
WARNING! Long post incoming. Summary: Added a sound engine for sound fx playback, currently trying to implement a device manager. For the last few days I've been focused on implementing a device manager plugin, to provide information about the various devices connected to the system. This is where I've come across the first major 'cross-platform' issue with the software. Qt doesn't provide classes/methods for this sort of functionality (and it's a little ridiculous to assume that it would). For now, since I'm developing on Linux, I'm focusing on trying to finish the alpha on this platform, and I'll try and fill in the major differences (the device manager being the main one I can think of) on other platforms. The device manager should manage and inform other plugins of devices relevant to Prismatic. I originally preferred the idea that plugins handle devices on their own (and this is still a valid option for any plugin that doesn't want to use the device manager), but when I started adding the diagnostics stuff, I realized that there was no good way of finding out whether the necessary OBDII device was connected, where it was located, etc. To complicate things further, I use a USB OBDII cable which uses an extremely common USB-RS232 controller... so common, that it's used on my GPS device as well. To the system, the devices just show up as USB serial devices, with no identifying marks for Prismatic to auto-detect what kind of device each is. It should be noted it's possible for each plugin to try and detect what a device is by trying to send messages to the device and waiting for a response, but this is tedious and time consuming. For instance, for an OBD plugin that uses the ELM327 controller, you'd have to check several protocols (ISO-9141-2, SAE J1850, CAN, etc) at multiple baud rates for every device that might be the OBDII accessory. To circumvent this, software that use such devices usually give the user the option to select a COM port (Windows) or tty device (Linux). This isn't really acceptable for a front-end though -- why should the end user have to know what address the system internally assigns a device just to use it? I tried to find some middle ground in this problem. Right now, my idea is to ask the user to identify what kind of device something they plug in or pair with is by selecting from a predetermined list. Once a device has been identified, Prismatic can auto-detect it every time it's used in the future. The device manager plugin can then be used to notify any other plugins that want to know if a specific device exists, or has been added or removed from the system. Implementing this isn't too straightforward once you consider that there are a lot of different devices and methods with which they connect to a system. Storage, OBD, GPS, Phones are all connected with any combination of wi-fi, USB, bluetooth, etc. For now I've selected a subset of devices and connection types to implement. This list will grow in the future. Another small update that I managed to work in was the implementation of a sound engine. This can be used to play back sound effects for pretty much anything from button clicks to (maybe one day) voice based navigation. The sound engine has its own volume control, just like the music engine so that they can be adjusted relative to each other. I hope I can get the device manager working well soon... I'm pretty psyched about starting on the diagnostics stuff (that's why I started this project in the first place!). -Pris Add Comment FileManager and Music Plugin changes 12/08/2011
Over the last few days, I made a some improvements to the music plugins and changed the FileManager a bit. The music library plugin uses sqlite to store data for a music collection. I changed the library plugin to do all database queries and modifications in one transaction which remains open for the entire duration of program execution, rather then commit a transaction for each mod. This means that any changes that now occur to the database occur and stay in memory up until prismatic is shut down, at which point they're committed to disk. I also improved the logic for importing album art. Both of these changes result in a lot less time taken to index music. The other music changes are for the music player: I optimized the music player plugin's visualizer in the UI, and cleaned up the appearance of the playlist view. I also spent some time modifying the file manager. I used a lib called QDrive to add support for detecting and accessing external media, like USB keys and SD cards. It works pretty well, but I'm having some issues with automatic detection of media insertion and removal... I hope I can fix these issues soon. The final change is mostly a cosmetic one; I just added a dialog to the file manager for user confirmation before deleting files. -Pris     MusicEngine Updates 12/04/2011
Quick update with the MusicEngine plugin... I finally added a settings pane for equalizer and volume control. I also added a couple of UI components to facilitate creating other panels that might require similar behaviour. -Pris   Status Update 12/03/2011
I really should try to update more than once a month! There have been a lot of refinements and bugfixes between this and the last update. Unfortunately I still haven't started on the diagnostic plugins (I'm not too worried about implementing that yet, even though I'll probably end up eating my words in the near future). It's not practical to go through all the changes in detail, so I've briefly summarized some points: * A ridiculous number of updates to the underlying UI components which makes designing the plugin UI much easier * Improved album art support in the music library, roughed in music visualizer * Plugin settings UI implementation now offloaded to each plugin instead of auto-generating a settings page (compromised simplicity for additional functionality) * Roughed in a plugin for simple file management which will ideally be used to transfer music, maps, logs and other data between different media * Customized libosmscout for more complex map styling options (trade-off between map rendering speed and map image quality). Created an offshoot map paint engine to handle map label intersections, one-way roads and text outlines I still have a huge To Do list with respect to prismatic's current plugins, which I'm slowly chipping away at... as of right now there are 9 separate plugins!: * PrisFileManager - simple file manager * PrisHome - home screen, links to all other plugins * PrisMusicBrowser - music collection browser * PrisMusicEngine - music engine backend * PrisMusicLibrary - music library database * PrisMusicPlayer - music playback control * PrisNavEngine - nav engine backend (maps + search) * PrisNavGps - nav gps provider * PrisNavViewer - nav map and search viewer I want to make sure that the above plugins are robust and as bug free as possible before packaging an alpha version of prismatic. I'll try to update more frequently in the future (especially as I get started on the diagnostics and cell phone stuff). -Pris                     Plugin Development and Music Player updates 11/05/2011
It's been awhile since the last update. There have been some underlying changes for all of prismatic that make development a bit easier. Now when creating a plugin, you can link against three separate libs: PrisSettings, PrisLog and PrisTypes. PrisSettings gives you an easy way to store and access settings (strings only). PrisLog lets the plugin output to the application's log file to help with debugging and PrisTypes gives access to the data types shared across plugins. I also added a PluginHelper object to the plugin interface that exposes some basic functionality like changing the page that's being displayed, adjusting the volume, or accessing settings from the QML engine. One issue I've run into with respect to plugin development is exposing the signals, slots and properties that each plugin implements. I still need to figure out a good way to inform a developer what functionality they have access to from other possibly closed source plugins. I've also switched over to GStreamer for the music back end. This took a lot longer than I thought it would, but everything seems to be in okay shape now. Compared to the previous iteration, the new music engine has: * Gapless Playback * Equalizer * Visualizer Support It should also be fairly easy to add features in the future. One thing I'd like to see is balance control. Most cars have right/left and front/rear balance adjustment by default, but I'm not sure exactly how to do this through Prismatic. I think it depends on the sound card of the system and the audio setup of the car. I'll should be able to finish music playback in the next few days. After that, it's on to revamping OBD, which I've already more or less implemented before and then Cell Phone support through bluetooth. Mapping / Navigation nearing completion 10/13/2011
Hiya, I've made some decent updates to mapping / navigation. Right now, Prismatic has the ability to: * Search by Name * Search by Type (Fast Food, Cafes, Parks, etc) * Search by Address (Streets, Street No., and Intersections) The search works reasonably well and is lightning quick. The search relies on data in OpenStreetMaps being tagged a certain way. Since by nature OpenStreetMaps is an open initiative, with much of the data filled in by individuals, the tag data isn't guaranteed to meet a certain standard which can introduce issues when searching. However, a lot of the data follows set guidelines, and so most of the map data provides good results. After performing a search, the user has the option to display it on the map. Prismatic will automatically set the map zoom and focus to include both the user's position and the target (if an active GPS device is present). I still need to work on some map drawing issues with the underlying map library (libosmscout) and refine the types of data Prismatic makes searchable. The UI for maps and the search tool also needs a bit of tweaking, which I'll probably take care of closer when I make final touch ups for the alpha release. I haven't made any mention of routing yet... I decided to put it on the back burner. libosmscout does have some routing functionality, but I haven't explored it at all yet, and I feel I'll need to invest a decent amount of time to add it to Prismatic. There's a good chance routing won't be in the alpha release. -Pris 
Updates on map searching and music 10/03/2011
Hiya, The mapping library I'm using (libosmscout) has undergone a few important changes, and a couple of bugs have been fixed as well. It seems like it'll be more than adequate for performing map searches... the searches are also much *much* faster than the method I was using previously. I should have a pretty solid map rendering + map searching base setup in Prismatic by the next update which is pretty encouraging. I also decided that the music back end I was using (Qt-Mobility's multimedia kit) was too simplistic. It was really easy to implement, but there's no control on how audio is played back excluding volume. This means no easy way to implement an equalizer, visualization, etc. Qt has it's own low level audio tools as well, but they seemed a little too low-level. After searching around a bit, I chose to switch over to GStreamer as Prismatic's media back end, which Qt-Mobility uses internally as well. There's a neat Qt wrapper lib for gstramer (aptly named "Qt-GStreamer") which I'll be using to implement the media engine. GStreamer has a ton of plugins (codecs, equalizers and the like) that will allow for a lot more features and extensibility for media in Prismatic. Theoretically it's even possible to implement video capture (rear-view camera, anyone?) without too much trouble! But that's not really on the drawing board right now. -Pris Status Updates 09/26/2011
Hiya, I've run into some roadblocks with mapping and navigation and haven't really made much progress. I had a few issues with the library I was using to implement a search on mapdata (Osmium) and I couldn't figure out how to proceed. I think there's a bug when the library writes to file as it corrupts the data when some unknown threshold for writing tag data is reached. I tried to get help with this, but haven't heard back from the library devs yet. I think that I'm using the library in a really unorthodox way so I'm not expecting much support. So now I'm trying to use libosmscout (which I originally wanted to use for map rendering only) to search as well. I'm sure I can use this library to search through the map, but I get some strange behaviour when trying to get data. There's a good chance I'm just using the library incorrectly but there aren't any examples that do what I want and the library itself isn't documented too well. That being said, I've been in contact with the author since I began evaluating libosmscout and he's been extremely helpful with everything, so I think this is a minor problem (even though I'm stuck right now). I've already conceptually implemented searching for regions (locales, suburbs, etc), for point of interests by name and category (ie. search for Shopping, or for the "Eaton Centre"), and searching for streets and street addresses (ie. search for "King Street" or "700 King Street West"). I also added GPS support using a gpsd wrapper for some Qt Mobility components. I'm not sure how this is going to work out on Windows yet, but it works great on Linux. As would be expected, the map automatically tracks along with the current GPS position. The only thing left for navigation after I fix the search is routing and directions. I think I might hold off on implementing that until later. Other than that, there have been some UI improvements on the developer end -- pretty much every component of the UI has a custom version that's much easier to plug into any given QML Panel for use with Prismatic. The custom components have consistent formatting and additional helper functions. For instance, ListViews and other large content now have a scrollbar and dedicated buttons so scroll up and down. -Pris   I implemented a brute-force search method like I detailed previously. I'll go over some of the details of the implementation in this post. I used Osmium, a toolkit for manipulating OpenStreetMap data to split a single OpenStreetMaps file (*.osm.pbf) into several tiles. The tiles are encoded using quadkeys down to a desired resolution. The corresponding bounding box for each tile and its quadkey is then stored in an sqlite database. When splitting up the data, nodes (point geometries) are easy to organize, but deciding which tiles ways (line geometries) go in isn't as straightforward. I chose to store ways in the tiles that contained their averaged lat/lon coordinates. I also saved the node coordinates of the way within the way's tag data. This takes up extra space, but gives the search tool all of the way's geometry without having to passing through the data multiple times. To perform a lookup, the user needs to specify a region first... this can be a city, district, suburb, etc. The location of the region is looked up in the index database. The lat/lon coordinates of the region are then used to build a search queue of the data tiles spatially outward from the specified location. There's a distance bounding box limit in place (I think it's set to ~25km right now) to prevent the search from going on forever. So far, you can search for something by name, or category. You can lookup a street as well, but I'm still working on searching for a full street address (ie. w/ street number). In terms of lookup time, the best case performance is great and the worst case performance is poor. Exhaustively looking for something specific takes the most time. Here are some examples: Lookup all instances of "eaton" by name, starting out from Toronto, searching for 25km takes ~8 seconds to find: ------------------------------------------- Result # 0 ------------------------------------------- Name: "Eaton Centre" Desc: "" SName: "" SNum: -1 LocR: "Toronto" ExtR: "Toronto" Lat: 43.6536 Lon: -79.3826 ------------------------------------------- Lookup bars in Toronto limited to 25 search results, takes less than a second: ... ------------------------------------------- Result # 23 ------------------------------------------- Name: "The Wheat Sheaf" Desc: "Bar/Pub" SName: "" SNum: -1 LocR: "Fashion District" ExtR: "Toronto" Lat: 43.6437 Lon: -79.4029 ------------------------------------------- ------------------------------------------- Result # 24 ------------------------------------------- Name: "Friends Bar & Grill" Desc: "Bar/Pub" SName: "" SNum: -1 LocR: "Cabbagetown" ExtR: "Toronto" Lat: 43.6583 Lon: -79.3722 ------------------------------------------- ------------------------------------------- Result # 25 ------------------------------------------- Name: "O'Gradys" Desc: "Bar/Pub" SName: "" SNum: -1 LocR: "Church-Wellesley Village" ExtR: "Toronto" Lat: 43.6645 Lon: -79.3806 I left a lot of unnecessary information in the tile data that is split from the single OpenStreetMap file. I hope that I can notably improve search speed if I take this information out. I'll end this post with some images of map rendering with libosmscout, another toolkit for working with OSM data. -Pris    I thought I'd make a post about what I've encountered trying to implement a simple offline search for OpenStreetMap data. Searching through OSM data involves going through all of the map's nodes, ways and relations, and their tags to look for something specific. However, with a large dataset, there are millions of nodes and ways, so searching for something in a reasonable time frame can be a challenge. I started out by using a toolkit called Osmium to save the information I wanted from a map file to an sqlite database so I could organize and search the data. The map I used contained Toronto -- a decent sized city with a lot of information, but with a compressed file size of only ~17mb! Importing the data into sqlite and running queries on it to search for specific things worked well. I used a bounding box to speed up the search a little bit, and overall the speed of running a search and getting results was pretty good. However, the file size of the database was around 90mb. That's over 4x the size of the original file! For someone who wanted a data set for all of Canada, (and assuming the database size scales linearly), that would mean an increase from ~800mb for the map, to around 3.2gb, just for a database to search! After trying to figure out how to proceed, I came up with an idea that might let me keep the original map file and bypass the database altogether (for the most part). Instead of importing the map data into an sqlite database, I'm going to search the map file itself using Osmium. Searching through every node in the Toronto map file with Osmium directly took my computer about 5 seconds. My computer's a 3-4 year old laptop, so I feel the search time is pretty good. The problem with this method is that searching nodes is a 'brute-force' method -- I can't use an index like in a database, so as the file size of the map increases, the time will go up as well. Reading a much bigger file isn't feasible. My idea is to chop up a large map file into a grid, and then keep an index of what goes where with a small database file. I'll include major regions in the database file (cities, counties, etc), so the first step of a search will be to figure out which grid tile I want to perform a full search on. Once that's done, a brute-force search shouldn't be too expensive, especially if the tile sizes are kept small. |
Create a free website with
Weebly
