More advance feature support

[yepher]

Yanoushek,

I've gotten a little addicted to this project. I hope you don't mind but I am going to try and implement the features on the roadmap. My first step will be to refactor adb out of the AppDelegate class. I will do this by creating a LogcatDatasource class and trying to move filtering, search, adb etc into it. I think this will make things a lot easier in the long run. Maybe I will pull the to tableviews into their own class too.?.

Here is the list of items on the roadmap and a few comments about them:

1. Support for more than 1 device at a time.

I will start by allowing user to select from a list of devices (if multiple connected) and view the log for the selected device. How did you envision this to work? I foresee having one window per device log (maybe I can hack together a demo of this after the refactor). Or being able to select a threadID and open its view into another window. This way you could look at two threads side by side.

2. Automatically generated filters for all the tags, types etc.

I added ability to add a filter by right clicking on the row/column. In my opinion this is better than auto generating filters. I do think it would be useful to be able to import and export filters. If we start using predicates this will probably need to get a lot smarter.

3. Better and easier filter definitions with NSPredicateEditor.

I've worked with NSPredicate a few times. As I do the refactor I will consider changing to NSPredicate for the filtering

4. RegExp searching and filtering.

No plan to solve this during this refactor

5. Autodetection of adb location (or bundling the adb within LogCat?).

Is this still a plan since ADB is packaged with the app? I could see an argument for preferring the users ADB and if it can't be found than use the pre packaged one.

6. A better solution for reading and parsing adb output.

I think it will be a lot more efficient if we use "adb logcat -v threadtime" and parse with NSScanner. I am going to test this out and see what the most efficient way to do this is. I wan to optimize the initial launch because it take several seconds before we get through the cached log.

[YANOUSHek]

Hi Chris,

thanks for the kind words and I'm happy that someone shares my addiction to this little app. Unfortunately, I can't spend as much time as I wanted with this little project, so I'm really excited that you came around :)

Refactoring adb is a good start but I wouldn't move the filtering into there. My whole idea was to have a "model" layer that would be responsible for parsing all the logcat output into NSArrays and then let the window controller do it's filtering depending on the users actions.

I'll try to tell you my ideas about the roadmap but no matter what you choose to do it will be great for Android devs :)

Support more than 1 device

Supporting more than one device is a tricky thing and I'm not sure if it's possible to have it done the way I planned to. The idea was to have the parser always parse the output from all devices so that the data is always there for you and you could swap the view between devices using a popup menu in the toolbar. This way you won't have to open multiple windows to see output from all your connected devices but you could switch on the go when you needed to. As I wrote, I'm not exactly sure how to achieve that – the only thing that comes to mind is polling "adb devices" and reacting to changes but that might be really hard to do.

I was also thinking about allowing the user to have multiple LogCat windows running where you could select different devices (or even the same device so you could observe different filters on it). Let me explain this a little better by giving an example.

I used to work with big, complicated projects which did a lot of communications between different processes (let's call it integrating multiple apps). Each app had it's own set of tags set up for logcat which made it impossible to easily filter all of them so I was left with 2 options: observe only one tag or observe all the messages. It would be fantastic if I could see tag "X" in one window and tag "Y" in another. This way I only get what I want to see and I don't have to visually scan all the messages printed by logcat.

Automatically generated filters

I was thinking about splitting the sidebar into three groups expandable groups:

1. All messages – this would show all messages that are available for a device.
2. Custom filters – this would list all user defined filters which, at least for me, are the most important.
3. Automatic filters – this would list all the automatically generated filters for each of the tags, applications etc.

This is not the most important feature so I would move it to the bottom of the list.

NSPredicateEditor

I haven't worked with NSPredicateEditor and that's why I chose to do a simple form that lets you filter the way it's done now. I think that NSPredicateEditor can allow for an easy way to create really complex filter logic and that can be really useful.

Autodetection of adb location

I'd also push this down the list since the bundled adb seems to be working fine. I guess that using the users' installed adb might become troublesome when it's output format changes.

Better adb output parsing

This is a long one. If you plan to switch to "adb logcat -v threadtime" please make sure that it works on a couple devices. When I started working on LogCat I was shocked seeing how different devices output data to adb and "-v long" seems to be the only format that is consistent among different devices.

Either way the idea here was to remove the adb requirement all together by implementing the bare adb daemon that communicates with devices, but this is a hard thing to do. I did take a look at the code of adb and it's not that easy to get connected to the daemon running on the device and communicate with it so sticking to adb's output feels like the best solution for now.

I think that's it. Thanks again for offering your help and doing the great changes that are already a part of LogCat.

[yepher]

Thanks for the pull merge and for the the clarifications.

Thanks for the heads up on the log format variations. Do you have any specific devices that seemed to have an especially troublesome log format? I have about 12 different models that I can test with easily.

Have you looked at the binary log format at all adb logcat -B ? That seems like an especially good format for this kind of tool. Anyway I will be sure to try as many different kinds of devices as I can get my hands on.

Also,

Last night I was able to get most of the way through the LogDatasource refactor. It was easier to bring the filter over as well. Once I get to where the code is working again I am going to try and factor out the filter/search logic. Stay tuned.

[YANOUSHek]

I remember having the biggest problems with our old Samsung Galaxy Spica running Android 1.6 it's log output was completely different and only the -v long worked like it should. I did not know about the binary output – that might be the best way to do it. I'll test it out tomorrow on all our devices.

Can't wait to see your changes.

[yepher]

I'm not ready for a pull yet but I have multi device, multiple window (hack), and parsing with "-v threadtime". I've tested on 6 different models and it seems fine so far. It can get confused if you are disconnecting and reconnecting with different devices.

I added a switch in LogDatasource.m that lets you switch between the various logging types. I will give Binary parsing a try as well.

Here are the available options:
// 1 - adb logcat -v long
// 2 - adb logcat -v threadtime
// 3 - adb logcat -B # Work in progress
#define LOG_FORMAT 2

I found an issue with the "-v long" parser. It sometimes puts the message with the next date, tag, pid, etc. I am not sure if this is a bug I introduced in a previous change or if it was already there. I will try and sort that out.

I hacked in support for multiple windows and for multiple devices. For now I am just launching another instance of the app with "open -n LogCat.app". It is not pretty but it should give us a good feel for if the basic idea is a usable one. If/when we get real multi document support working it will definitely be a benefit not to have the filters in the LogDatasource (just like you said ;).

I need to focus on stabilizing now before trying to add in predicate support and put filters and search where they belong.

If you get a chance give it all a try and let me know if something is majorly broken. Especially with the "Galaxy Spica running Android 1.6". I wonder if it would be a good idea to checkin a bunch of test logs from as many devices as we can get.

[yepher]

I forgot to mention:

While thinking about parsing the binary log I was in need of a good way of reading/parsing a binary stream. I have quite a bit of code for doing this kind of thing but there is none I can provide to this project.

So I think I am going to use this for doing the binary parsing: https://gist.github.com/102331 do you have any better suggestions?

[yepher]

YANOUSHek,

I hope you are doing well. I noticed you have gone silent. I expect you are on holiday. If so, I hope you are having a good/relaxing time.

This is just a quick update incase you are monitoring things. I am avoiding pushing to github so your pull is easier. I could make another fork but this is easier for me [my git foo is weak ;) ].

First the bad news:

I implemented Binary log parsing which would be really great IF "abd logcat -B" did not insert random bytes of data in the message payload. I have some ideas that maybe I could still make it work but I am going to table it for now until I can understand the problem better. Once I get all the rest of the features working again I will check in the binary log filter code but commented out so we don't get compiler warnings for unused variables.

Now the good news:

Locally, Like you suggested in the first place I have removed filters and search from the datasource class (the next big step towards true multi device support). Now filters are 100% built using predicate editor. I have disable saving filters and searching but will start adding that back in now. All the users previously saved filters will no longer be valid. I am too lazy to write an upgrade routine that will convert old filters to new predicates. When they upgrade to the new version all the previously filters will just disappear, although they will still be in the save preferences. I did this by renaming the the value of KEY_PREFS_FILTERS.

Also, I think it will be better to change the "search" behavior especially now that filtering is so powerful. To me search should be the equivalent of "find". The table should jump to the next row that has the given search criteria.

Happy Holidays,
-- Chris

[YANOUSHek]

Hi Chris,

thanks for the update. I did have a Christmas break but now I'm back. Hope you had some rest during Christmas and that Santa was generous :)

I do like your changes. The predicates look awesome but they do not show up in the left pane and do not get saved. The current logcat parsing works great with all my devices.

[yepher]

Yanoushek,

Welcome back. Glad to hear you got some time off. Christmas for me was relaxing and Santa was very good to me.

I will look into the predicate saving issue. I assume you did give it a name and click the save button in the top right corner of the predicate editor?

If so, It maybe easier to identify the issue if you send me your:
- ~/Library/Preferences/com.splashsoftware.LogCat.plist

Also if you run it in XCode send me the information from the log window after trying to save a predicate.

-- Chris Wilson

[yepher]

Ok, today I've seen several folks that did not realize you have to press the "Save" button to save the filter. That is a mark of a bad UI. So now "OK" will save the filter instead of just closing the window. That seems to be the anticipated behavior.

See if the filter dialog works better for you now.

[yepher]

Yanoushek,

Happy New Year. I hope the year is starting off good for you.

Just a quick update. I could not live without the quick filter so now LogCat support both Find and Quick Filter.

-- Chris