For some time now Upstart has slowly been taking over the old SysV init scripts on most Linux distributions, for better or worse. Now some of the things they are trying to do with Upstart are good and have some merit, though at the cost of simplicity and ease of use. I fear the developers and supporters are worrying more about the 5% use case rather than the 95% use case and they have thrown the old mantra of keep it simple and stupid out the window. The problem when you give the 5% too much focus, you end up creating a solution that is horribly broken and resembles solutions from the little company up in Redmond Washington.
For those struggling to find your way through the new hell of Upstart here are a few pointers:
1. Dependency hell is brought to a whole new level of pain and frustration. Imagine a system where everything can emit and event and things “should” start or stop based on those events. Hey that is a great theory. But how on earth do you troubleshoot it. How do you get your system to start things in an exact order without building a chain like what we had back with SysV init scripts. In the old days of SysV, you gave all of your scripts a number, and they started in order from lowest to highest. If you wanted something to start sooner, you just lowered the number. Simple. Now you have to try and go through and figure out what starts when and what needs to happen before everything else. There is no tool that will print out a dependency map and there is no logging or debugging to speak of.
2. The location of Upstart configuration files can be found at /etc/init/. The old remaining SysV init scripts that can not yet be converted to Upstart can be found at /etc/init.d/.
3. The scripting support in the “script” blocks of Upstart is very limited and extremely basic. Basic things like echoing out data to the console does not work which makes it hard to try and use the scripts for anything other than shotgun starting a service or kill-9ing it.
4. The default answer to stopping a daemon in Upstart is to sent it a SIGTERM and wait a few seconds and if it is not stopped to send it a SIGKILL, a kill -9. Holy crap batman, there goes my database server that takes a while to shutdown cleanly.
5. You can not create any custom function blocks to pull extra status or details from a server that might support it. So your only option is to create a separate tool to manage the daemon. If you are going to do that, why not just stay with the SysV init scripts.
6. Most Linux Distributions are pushing this down our throats fast and hard, even when it is not fully baked or fully functional yet. This reminds me of the tactics that come from a little company in Redmond Washington.
7. Upstart can not manage all services yet, and some just do not start at all with Upstart. In the effort to build the most advanced leaky abstraction, some times a daemon will fork once or twice and Upstart can not track it and thus you get in to a state where Upstart “thinks” the service is stopped or started when it really has not. So imagine if that was your database server again, it would suck if the system went ahead and shutdown the filesystem even though the database server is not even close to shutting down.
I have spoken with the lead developer of Upstart several times. And he seems reasonable and desires to fix at least some of the problems and weaknesses in Upstart. However, I fear that there are some that he will not fix. Some of the things that we need it to do are not in the vision of it.
The things I would change about Upstart, sooner rather than later are:
1) Make the scripting engine inside the script blocks work with all script commands and be a fully functional scripting engine.
2) Provide a tool that prints out the exact order that things will start at boot with a parenthetical list next to each entry with its dependencies.
3) Provide the ability to add arbitrary functions/scripts to the Upstart configuration files so you can do something like “initctl apache super-details” and have it run a script block inside the configuration file called “super-details” which does what ever you need it to do.
I hope that Upstart and other proposed replacements to SysV init scripts can focus on making a better solution and not focus on trying to solve a 5% use case problem that causes pain and frustration for the other 95%.
Friday, October 29, 2010
Wednesday, October 6, 2010
PDF Document Manipulation
With the advent of advanced mobile reading devices that support full graphic displays, such as the iPad and soon to be released Android based tablets, we in the network security space find ourselves wanting to bring our entire reference library of books, ISO Standards, RFC, and diagrams with us everywhere we go without the concern of do we have internet connectivity and do we remember where all of the documents are without having to dig around the Internet.
The problem, however, is that a lot of the reference documents that we use are published on-line in text or PDF format and are only available in pieces (chapter by chapter). This was probably done to save bandwidth and for ease of downloading and viewing.
This type of on-line publishing can work well when using a full sized computer that has disk storage or a real file system and keyboard/mouse. This does not work well for mobile readers that do not have disk storage file system to store hundreds of separate files and may be used in places where you do not have internet access (on a plane over the Atlantic or in that basement corner conference room that is just out of wifi range).
We are going to address how we can manipulate on-line PDF documents that are published in parts to turn them in to one PDF “book” for easy of use on a mobile reader. We will also discussion in Example 2 how to fix bookmarks that no longer point to the right place due to adding or taking pages away.
Example 1:
We have found a book online that has 4 separate chapters that we would like to make a single PDF “book” out of for our mobile reader. Each chapter is a separate download called chapter1.pdf, chapter2.pdf, etc. The first page of each PDF is actually a blank page and the second page is note/summary/title page, neither of these pages we want to keep.
Step 1.1:
Download all related files to your computer. You will need to have local copies of all the files for the book or standard document in question. So lets download them to say /home/pdf/
Step 1.2:
Remove any leading or artifact pages that are not relevant to the combined book. Some times you will find that PDF documents have a leading blank page, or leader “summary” page, or master title page for every chapter. When this is not desirable, we can easily remove them prior to stitching the chapters together. (Be mindful of any copyright or usage rules when making changes). The tool we will be using is called “pdftk”.
What this command will do is take the current PDF of “chapter1.pdf” and create a new PDF document called “new-chap1.pdf” but it will skip the first two pages. It will start on page 3 and go to the end of the PDF document. Lets do this for the other 3 chapters as well.
Step 1.3:
Now that we have removed all of the leading white pages and title pages, let us combine the 4 chapters in to one PDF document called “book.pdf”
Step 1.4:
Now that we have a single PDF document that has all the chapters stitched together, we can now look in to adding PDF bookmarks to the file to make jumping around easier on our mobile reader. This can easily be done with a tool called “jpdfbookmarks” that you can download here: http://sourceforge.net/projects/jpdfbookmarks/. As of this writing I am using version 2.4.1. From this tool you can add bookmarks and sub-bookmarks very easily from their graphical interface. We will create a bookmark for Chapter 1, Chapter 2, etc, and we will also create sub-bookmarks called Chapter 1.1, Chapter 1.2, etc for all of the sub elements in the chapter. Once you are done save your changes.
Example 2:
We have a PDF document that we either created in Example 1 or that we downloaded from the Internet that we would like to add a title page or a picture page (picture of the cover of the book) to the front of the PDF document. Now ideally if this is from Example I, we would have done this during Step 1.3 so as not to mess up all of our bookmarks. If we try to just use the pdftk command by itself, all of our bookmarks will be off by the number of pages that we insert. This method would also work if we needed to remove a page and wanted to keep all of our bookmarks. So what we can do is the following:
Step 2.1:
Create our title page as a PDF document or convert the PNG/JPG picture of the book to a PDF document and call it “cover.pdf”.
Step 2.2:
Lets add the “cover.pdf” to the first of the “book.pdf” document. NOTE: when we do this all of the bookmarks will be off by the number of pages that we insert, but I will show you how to fix this. For the sake of explanation, let assume that the “cover.pdf” is only 1 page long.
This will create a new PDF document called “bookwithcover.pdf” that will have the new title page or picture cover added to the front of the book. For our example, all of the bookmarks will now be off by one page.
Step 2.3
First we need to export our current bookmarks so we can fix all of them in mass. If we use the “jpdfbookmarks” tool that we used up in Step 1.4 we can “dump”/export all of the current bookmarks for the “bookwithcover.pdf” document to a text file. This file will look like:
The “BookmarkName???” is just the name that you gave the bookmark with you created it and is what shows up in the bookmarks pane in your PDF viewer. The slash “/” # that follows the name is the page in the PDF document that it points to.
Step 2.4
We can now use a short piece of PERL to go in and fix all of the bookmarks in mass. This assumes that you saved the bookmarks as “dump.txt” and the changes will be saved to “dump1.txt”. We will also be changing all of the bookmarks by one page since we only added one page. NOTE: this is written in long hand PERL for readability.
Step 2.5
Now that we have a file called “dump1.txt” that has all of the corrected bookmarks, we need to import that back into our “bookwithcover.pdf” document. Let’s use the “jpdfbookmarks” tool once again to do this for us. Open the PDF document with jpdfbookmarks and then use the “Load” function in the “Tools” menu to load “dump1.txt” which is the new bookmarks. Save the PDF and you are done.
The problem, however, is that a lot of the reference documents that we use are published on-line in text or PDF format and are only available in pieces (chapter by chapter). This was probably done to save bandwidth and for ease of downloading and viewing.
This type of on-line publishing can work well when using a full sized computer that has disk storage or a real file system and keyboard/mouse. This does not work well for mobile readers that do not have disk storage file system to store hundreds of separate files and may be used in places where you do not have internet access (on a plane over the Atlantic or in that basement corner conference room that is just out of wifi range).
We are going to address how we can manipulate on-line PDF documents that are published in parts to turn them in to one PDF “book” for easy of use on a mobile reader. We will also discussion in Example 2 how to fix bookmarks that no longer point to the right place due to adding or taking pages away.
Example 1:
We have found a book online that has 4 separate chapters that we would like to make a single PDF “book” out of for our mobile reader. Each chapter is a separate download called chapter1.pdf, chapter2.pdf, etc. The first page of each PDF is actually a blank page and the second page is note/summary/title page, neither of these pages we want to keep.
Step 1.1:
Download all related files to your computer. You will need to have local copies of all the files for the book or standard document in question. So lets download them to say /home/pdf/
Step 1.2:
Remove any leading or artifact pages that are not relevant to the combined book. Some times you will find that PDF documents have a leading blank page, or leader “summary” page, or master title page for every chapter. When this is not desirable, we can easily remove them prior to stitching the chapters together. (Be mindful of any copyright or usage rules when making changes). The tool we will be using is called “pdftk”.
[jordan]:/home/pdf-> pdftk chapter1.pdf cat 3-end output new-chap1.pdfWhat this command will do is take the current PDF of “chapter1.pdf” and create a new PDF document called “new-chap1.pdf” but it will skip the first two pages. It will start on page 3 and go to the end of the PDF document. Lets do this for the other 3 chapters as well.
[jordan]:/home/pdf-> pdftk chapter2.pdf cat 3-end output new-chap2.pdf
[jordan]:/home/pdf-> pdftk chapter3.pdf cat 3-end output new-chap3.pdf
[jordan]:/home/pdf-> pdftk chapter4.pdf cat 3-end output new-chap4.pdf
Step 1.3:
Now that we have removed all of the leading white pages and title pages, let us combine the 4 chapters in to one PDF document called “book.pdf”
[jordan]:/home/pdf-> pdftk new-chap1.pdf new-chap2.pdf new-chap3.pdf new-chap4.pdf cat book.pdf
Step 1.4:
Now that we have a single PDF document that has all the chapters stitched together, we can now look in to adding PDF bookmarks to the file to make jumping around easier on our mobile reader. This can easily be done with a tool called “jpdfbookmarks” that you can download here: http://sourceforge.net/projects/jpdfbookmarks/. As of this writing I am using version 2.4.1. From this tool you can add bookmarks and sub-bookmarks very easily from their graphical interface. We will create a bookmark for Chapter 1, Chapter 2, etc, and we will also create sub-bookmarks called Chapter 1.1, Chapter 1.2, etc for all of the sub elements in the chapter. Once you are done save your changes.
Example 2:
We have a PDF document that we either created in Example 1 or that we downloaded from the Internet that we would like to add a title page or a picture page (picture of the cover of the book) to the front of the PDF document. Now ideally if this is from Example I, we would have done this during Step 1.3 so as not to mess up all of our bookmarks. If we try to just use the pdftk command by itself, all of our bookmarks will be off by the number of pages that we insert. This method would also work if we needed to remove a page and wanted to keep all of our bookmarks. So what we can do is the following:
Step 2.1:
Create our title page as a PDF document or convert the PNG/JPG picture of the book to a PDF document and call it “cover.pdf”.
Step 2.2:
Lets add the “cover.pdf” to the first of the “book.pdf” document. NOTE: when we do this all of the bookmarks will be off by the number of pages that we insert, but I will show you how to fix this. For the sake of explanation, let assume that the “cover.pdf” is only 1 page long.
[jordan]:/home/pdf-> pdftk cover.pdf book.pdf cat output bookwithcover.pdf
This will create a new PDF document called “bookwithcover.pdf” that will have the new title page or picture cover added to the front of the book. For our example, all of the bookmarks will now be off by one page.
Step 2.3
First we need to export our current bookmarks so we can fix all of them in mass. If we use the “jpdfbookmarks” tool that we used up in Step 1.4 we can “dump”/export all of the current bookmarks for the “bookwithcover.pdf” document to a text file. This file will look like:
Chapter 1 - BookmarkNameAAA/1,Black,notBold,notItalic,closed,FitPage
1.1 BookmarkNameBBB/1,Black,notBold,notItalic,open,FitPage
1.2 BookmarkNameCCC/2,Black,notBold,notItalic,open,FitPage
Chapter 2 - BookmarkNameDDD/3,Black,notBold,notItalic,closed,FitPage
1.1 BookmarkNameEEE/3,Black,notBold,notItalic,open,FitPage
1.2 BookmarkNameFFF/4,Black,notBold,notItalic,open,FitPage
etc….
The “BookmarkName???” is just the name that you gave the bookmark with you created it and is what shows up in the bookmarks pane in your PDF viewer. The slash “/” # that follows the name is the page in the PDF document that it points to.
Step 2.4
We can now use a short piece of PERL to go in and fix all of the bookmarks in mass. This assumes that you saved the bookmarks as “dump.txt” and the changes will be saved to “dump1.txt”. We will also be changing all of the bookmarks by one page since we only added one page. NOTE: this is written in long hand PERL for readability.
#!/usr/bin/perl
$file="dump.txt";
open (DATAIN, $file);
@fileline = ;
close (DATAIN);
$file1="dump1.txt";
open (DATAOUT, ">$file1");
foreach (@fileline)
{
m/\/(.{1,3})\,/;
# This is where we increase it by 1.
$newpage = $1 + 1;
s/\/.{1,3}\,/\/$newpage\,/;
print DATAOUT "$_";
}
close (DATAOUT);
Step 2.5
Now that we have a file called “dump1.txt” that has all of the corrected bookmarks, we need to import that back into our “bookwithcover.pdf” document. Let’s use the “jpdfbookmarks” tool once again to do this for us. Open the PDF document with jpdfbookmarks and then use the “Load” function in the “Tools” menu to load “dump1.txt” which is the new bookmarks. Save the PDF and you are done.
Thursday, June 17, 2010
Music Editor
Transcribing musical scores from one key to another or from one instrument to another can be painful and somewhat time consuming. While playing the Cello, I find myself needing to often transcribe music from Violin down to Cello so I can play it as a solo Cello piece. However, finding a musical note editor that allows you do this easily and cheaply is difficult.
For the past few months I have spent part of my research time each week searching for a good easy to use tool that was highly flexibly. I have looked at tools for Linux, tools that run on Mac, and also tools that run on Windows. There are lots of great tools out there, and it seems like a lot of musicians love and use either Sibelius (http://www.sibelius.com/) or Finale (http://www.finalemusic.com/). Both of these are great tools, though somewhat restrictive and very expensive.
Most of the Linux community uses and talks about MuseScrore, NtEd, NoteEdit, Rosegarden, Lilypond and a few others. But in the traditional Linux world, most of these tools are either somewhat unstable, lack core features, or are just not intuitive and easy to use.
None of these tools really worked for me, and the ones that got close, were very expensive. The problem was I was looking for a tool that would allow me to move stuff around the score as I needed, something that would allow me to expand a measure or add any type of extra notation details that I wanted to add. Even the idea of just writing arbitrary text to the score proved to be impossible with most tools. I hated the idea of writing the score, printing it out, and then hand writing extra details. I wanted to be able to do it all on the original and have it print out all nice and pretty. Some of the basic things I wanted to be able to do were: 1) add Suzuki style fingering, 2) add notation to keep track of which string I was planning on playing the note on, and 3) add some details to help me remember when I needed to shift or extend and which position it was in.
After trying most of the tools out there I gave up and resorted to the idea that I would just do this the very long way and do it in graphics program like Photoshop. But then I had the idea that I could just do this in Visio if I had the stencils. It would be pretty easy to make the stencil I needed, it would just take a lot of time. But once I had them it would be super easy to create and edit my scores since I use Visio nearly every day at work and love the flexibility that it offers. So I went looking on the Internet and found a site that had already done a lot of the work for me. Colleen Kobe has created a whole set of Visio stencils for musical score editing. These stencils are wonderful and I will say that the $30.00 she charges is well worth it. I bought the stencils and have been so happy with what I can do with them.
The Visio stencils I bought are called “Staff Scribe” and can be purchased at http://www.colleenkobestudios.com/music.html. After playing around with creating some scores, I love the flexibility that Visio gives. Now some users might not care for the granular controls or the extra flexibility but it works for me. Also some users may complain that it takes a little too long to do basic things, my advise to that is that as you get more comfortable with Visio, it will be faster.I Colleen has done a wonderful job building these stencils and I would suggest you get them if you have a copy of Visio and are comfortable using Visio.
After using the Staff Scribe stencil kit for a few days I noticed that a few things were missing and have since created my own add-on stencil to this kit that you can download from here for free. This stencil is designed to be used with Colleen’s Staff Scribe stencils and is not much use with out it. Also I followed her her naming convention I have called my stencil “Staff Scribe--Extras.vss”.
I am also attaching some Cello major scale and arpeggio fingering guides that I have recently put together to help young Cello players learn their scales, the notes, and the fingerings. Enjoy.
For the past few months I have spent part of my research time each week searching for a good easy to use tool that was highly flexibly. I have looked at tools for Linux, tools that run on Mac, and also tools that run on Windows. There are lots of great tools out there, and it seems like a lot of musicians love and use either Sibelius (http://www.sibelius.com/) or Finale (http://www.finalemusic.com/). Both of these are great tools, though somewhat restrictive and very expensive.
Most of the Linux community uses and talks about MuseScrore, NtEd, NoteEdit, Rosegarden, Lilypond and a few others. But in the traditional Linux world, most of these tools are either somewhat unstable, lack core features, or are just not intuitive and easy to use.
None of these tools really worked for me, and the ones that got close, were very expensive. The problem was I was looking for a tool that would allow me to move stuff around the score as I needed, something that would allow me to expand a measure or add any type of extra notation details that I wanted to add. Even the idea of just writing arbitrary text to the score proved to be impossible with most tools. I hated the idea of writing the score, printing it out, and then hand writing extra details. I wanted to be able to do it all on the original and have it print out all nice and pretty. Some of the basic things I wanted to be able to do were: 1) add Suzuki style fingering, 2) add notation to keep track of which string I was planning on playing the note on, and 3) add some details to help me remember when I needed to shift or extend and which position it was in.
After trying most of the tools out there I gave up and resorted to the idea that I would just do this the very long way and do it in graphics program like Photoshop. But then I had the idea that I could just do this in Visio if I had the stencils. It would be pretty easy to make the stencil I needed, it would just take a lot of time. But once I had them it would be super easy to create and edit my scores since I use Visio nearly every day at work and love the flexibility that it offers. So I went looking on the Internet and found a site that had already done a lot of the work for me. Colleen Kobe has created a whole set of Visio stencils for musical score editing. These stencils are wonderful and I will say that the $30.00 she charges is well worth it. I bought the stencils and have been so happy with what I can do with them.
The Visio stencils I bought are called “Staff Scribe” and can be purchased at http://www.colleenkobestudios.com/music.html. After playing around with creating some scores, I love the flexibility that Visio gives. Now some users might not care for the granular controls or the extra flexibility but it works for me. Also some users may complain that it takes a little too long to do basic things, my advise to that is that as you get more comfortable with Visio, it will be faster.I Colleen has done a wonderful job building these stencils and I would suggest you get them if you have a copy of Visio and are comfortable using Visio.
After using the Staff Scribe stencil kit for a few days I noticed that a few things were missing and have since created my own add-on stencil to this kit that you can download from here for free. This stencil is designed to be used with Colleen’s Staff Scribe stencils and is not much use with out it. Also I followed her her naming convention I have called my stencil “Staff Scribe--Extras.vss”.
I am also attaching some Cello major scale and arpeggio fingering guides that I have recently put together to help young Cello players learn their scales, the notes, and the fingerings. Enjoy.
Thursday, June 3, 2010
Much Ado About Android – part II
Before we start talking about hacking your Android phone and doing neat little things to it, we need to go over all of the bits and pieces that are needed. Now this series of posts is not designed to be a complete step by step for every Android device out there. It is however, designed to be a complete walk through of what I did to hack my HTC Magic. My hope is that this will give you the foundation from which you can gather the right information for your device. As we all know, trying to piece this information together from Internet forums can be painful. NOTE: You need to be really careful when modifying embedded devices like a phone. If you do things wrong, in the wrong order, or if you do not have enough patience you can brick your device. Bricking your device means that you have turned it in to a very expensive paper weight and that it is dead and unusable as an electronic device. Make sure you spend some time doing your research into issues that people are having with your specific phone and do not assume that just because it is an HTC Magic like mine that it is exactly the same as mine. Internal motherboards, chips, radios, etc can be different and you need to gather all of that information. Further, you do this at your own risk and your millage may very. I am not responsible in anyway for anything that you do or do not do with the information found in these posts. If you brick your phone, I am sorry, but it is not my fault.
Step 1 - The first thing that you are going to need to do is download and install the Android SDK (Software Developer Kit) for your OS. I will be doing all of my examples from Linux, but tutorials are out there for Windows and MAC. You can download the SDK from http://developer.android.com/sdk/index.html. Follow the install guide to get this setup. I installed mine at /local/android-sdk/
Step 2 - If you have an HTC device you will need to download the fastboot tool from the HTC developer site. This download can be found at: http://developer.htc.com/adp.html. Unzip this file to the /local/android-sdk/tools/ directory. You will need to “chmod 755 fastboot” to make it executable too. If HTC did not make your device, then you will need to find something similar from the vendor that made your phone.
Step 3 - Enable the “USB Debugging” feature on your device. You can do this by pressing: The MENU key -> Settings -> Applications -> Development and then enabling USB debugging. This is important as we want to tell the phone that we are going to be doing development on it and to expect commands to be delivered to the phone over the USB connection.
Step 4 - We need to tell our Linux USB stack how to deal with the HTC phone so that we can talk to it via the Android SDK and HTC fastboot tools. You can see documentation for this at: http://developer.android.com/guide/developing/device.html. For Ubuntu 9.04 and 9.10 you can do this by creating the following file: /etc/udev/rules.d/51-android.rules and adding the following to it:
SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"
chmod a+r /etc/udev/rules.d/51-android.rules
Step 5 – Gather build information about your phone. You can get this information by pressing: The MENU key -> Settings -> About phone. This information will help you when you need to go look for specific details about your device. My HTC Magic had the following from the factory.
HTC Magic
Firmware version: 1.5
Baseband version: 62.52S.20.18U_3.22.20.17
Kernel version: 2.6.27-357975db herbert@and18-2 #1069
Build number: 2.16.707.3 146733 CL#32934 release-keys
In my next post I will talk about ROMs, the various boot loaders and how to get more details about your phone, the type of motherboard it has, what kind of radio firmware is loaded, what kind of bootloader it has, etc. We will also get a recovery ROM loaded and installed.
Step 1 - The first thing that you are going to need to do is download and install the Android SDK (Software Developer Kit) for your OS. I will be doing all of my examples from Linux, but tutorials are out there for Windows and MAC. You can download the SDK from http://developer.android.com/sdk/index.html. Follow the install guide to get this setup. I installed mine at /local/android-sdk/
Step 2 - If you have an HTC device you will need to download the fastboot tool from the HTC developer site. This download can be found at: http://developer.htc.com/adp.html. Unzip this file to the /local/android-sdk/tools/ directory. You will need to “chmod 755 fastboot” to make it executable too. If HTC did not make your device, then you will need to find something similar from the vendor that made your phone.
Step 3 - Enable the “USB Debugging” feature on your device. You can do this by pressing: The MENU key -> Settings -> Applications -> Development and then enabling USB debugging. This is important as we want to tell the phone that we are going to be doing development on it and to expect commands to be delivered to the phone over the USB connection.
Step 4 - We need to tell our Linux USB stack how to deal with the HTC phone so that we can talk to it via the Android SDK and HTC fastboot tools. You can see documentation for this at: http://developer.android.com/guide/developing/device.html. For Ubuntu 9.04 and 9.10 you can do this by creating the following file: /etc/udev/rules.d/51-android.rules and adding the following to it:
SUBSYSTEM=="usb", SYSFS{idVendor}=="0bb4", MODE="0666"
chmod a+r /etc/udev/rules.d/51-android.rules
Step 5 – Gather build information about your phone. You can get this information by pressing: The MENU key -> Settings -> About phone. This information will help you when you need to go look for specific details about your device. My HTC Magic had the following from the factory.
HTC Magic
Firmware version: 1.5
Baseband version: 62.52S.20.18U_3.22.20.17
Kernel version: 2.6.27-357975db herbert@and18-2 #1069
Build number: 2.16.707.3 146733 CL#32934 release-keys
In my next post I will talk about ROMs, the various boot loaders and how to get more details about your phone, the type of motherboard it has, what kind of radio firmware is loaded, what kind of bootloader it has, etc. We will also get a recovery ROM loaded and installed.
Saturday, May 29, 2010
Much Ado About Android – part I
After years of dealing with completely frustrating and useless Windows mobile cellular devices I decide it was time to jump on the inevitable bandwagon of Google and their Android platform. The reason I choose Android over the iPhone is I like the idea of a little more freedom and while I do not agree with all of Google’s business practices, I appreciate that their prison is little more accommodating. Now, every technology platform is its own little prison, regardless of what people will even say about Linux or FreeBSD.
When it comes to vendors, hands down, Apple makes the nicest and most comfortable prisons. Microsoft makes functional prisons that appear to work seamlessly so long as you can tolerate all of the constant maintenance, damp infected rooms, and doors that should lead to bliss but only lead to dead ends. The Google prison is very similar to Apple’s but not quite as refined or clean. The trade off is that Google lets you out of your cell from time to time and even lets you share food and toys with your neighbors. And if you are adventurous enough and willing to go through all the hoops you can even leave the prison for short periods of time to go get your own soda and desert. All in all Apple and Google make very similar and wonderful prisons, but I choose Google Android.
Before I get carried away and you all think I am hook-line-and-sinker in love with Google; Google is evil and stores and tracks way too much information about what we do, what we like, and how we use our computers and the Internet. The reason I am okay with Google compared to the tactics that Microsoft employs is that Google does not force their technology down our throats. Now if Google decided that you could only use their search engine from certain computers, of that you could only access GMail through Chrome, or if they created their own language that you were forced to use to access any of your data stored on their cloud, I would definitely lump them in to the Microsoft camp and boycott them. But I digress; this post is not about my views of why and how Microsoft failed when they could have so easily taken over the world.
The phone I purchased was an unlocked version of the HTC Magic (same as the T-Mobile myTouch 3G, just an unlocked version). This phone shipped with version 1.5 of the Android OS commonly referred to by its code name of Cupcake (Version 1.6 has a code name of Donut for example, 2.0/2.1 is called Eclair, and 2.2 is called Froyo). Now as you can see, version 1.5 is really starting show its age. Eclair is out and Froyo is quickly on its way. So I decided I was going to take Google’s pledge to the test and try and take my phone out for a walk and see about updating its code.
So following all the buzz about newer features and the beauty of the Android platform I spent the better part of the weekend reading and reading and reading about all the bits and pieces that are needed to get your phone “hacked”, “rooted”, and upgraded. Now I must say that there is a LOT of information out there about how to do this and do that, and the folks over at http://forum.xda-developers.com and http://androidcommunity.com have a plethora of information. However, like all things Linux it is a PAIN-IN-THE-BACKSIDE to try and piece it all together. Linux people and developers are often of the mindset that it is it out there somewhere in some nasty forum list somewhere and that you need to read every meaningless post for hours and hours to find the one nugget of information that you need.
Over the next few days/weeks I am going to write a series of short posts that talk about the various bits and pieces and how they all bolt together so that you do not need to spend hours trying figure out what is what.
When it comes to vendors, hands down, Apple makes the nicest and most comfortable prisons. Microsoft makes functional prisons that appear to work seamlessly so long as you can tolerate all of the constant maintenance, damp infected rooms, and doors that should lead to bliss but only lead to dead ends. The Google prison is very similar to Apple’s but not quite as refined or clean. The trade off is that Google lets you out of your cell from time to time and even lets you share food and toys with your neighbors. And if you are adventurous enough and willing to go through all the hoops you can even leave the prison for short periods of time to go get your own soda and desert. All in all Apple and Google make very similar and wonderful prisons, but I choose Google Android.
Before I get carried away and you all think I am hook-line-and-sinker in love with Google; Google is evil and stores and tracks way too much information about what we do, what we like, and how we use our computers and the Internet. The reason I am okay with Google compared to the tactics that Microsoft employs is that Google does not force their technology down our throats. Now if Google decided that you could only use their search engine from certain computers, of that you could only access GMail through Chrome, or if they created their own language that you were forced to use to access any of your data stored on their cloud, I would definitely lump them in to the Microsoft camp and boycott them. But I digress; this post is not about my views of why and how Microsoft failed when they could have so easily taken over the world.
The phone I purchased was an unlocked version of the HTC Magic (same as the T-Mobile myTouch 3G, just an unlocked version). This phone shipped with version 1.5 of the Android OS commonly referred to by its code name of Cupcake (Version 1.6 has a code name of Donut for example, 2.0/2.1 is called Eclair, and 2.2 is called Froyo). Now as you can see, version 1.5 is really starting show its age. Eclair is out and Froyo is quickly on its way. So I decided I was going to take Google’s pledge to the test and try and take my phone out for a walk and see about updating its code.
So following all the buzz about newer features and the beauty of the Android platform I spent the better part of the weekend reading and reading and reading about all the bits and pieces that are needed to get your phone “hacked”, “rooted”, and upgraded. Now I must say that there is a LOT of information out there about how to do this and do that, and the folks over at http://forum.xda-developers.com and http://androidcommunity.com have a plethora of information. However, like all things Linux it is a PAIN-IN-THE-BACKSIDE to try and piece it all together. Linux people and developers are often of the mindset that it is it out there somewhere in some nasty forum list somewhere and that you need to read every meaningless post for hours and hours to find the one nugget of information that you need.
Over the next few days/weeks I am going to write a series of short posts that talk about the various bits and pieces and how they all bolt together so that you do not need to spend hours trying figure out what is what.
Friday, January 8, 2010
768 bit RSA is Broken!
In today's electronic world nearly all sensitive electronic communication is protected by a process that we commonly know as encryption. To the common user, encryption means SSL and SSL is all there is. In reality, SSL is only one tool in the entire encryption toolkit, and encryption is only one part of cryptography.
When we peel back the layers of the onion, modern encryption is all about mathematical hard problems. Meaning, that the mathematical functions are easy to process one way and very difficult to process in the reverse without a specific piece of information. Take for example the RSA algorithm that has been protecting internet e-commerce for years and is the common algorithm used for the creation of nearly all digital certificates. At the very foundation, RSA is about dealing with two very large prime numbers. These two large prime numbers are multiplied together to create a new number. This new number is then used in the algorithm to encrypted your data. Now the mathematical hard problem is that given the new number it is very difficult to factor that new number down to the two original prime numbers that created it. Fundamentally all that is protecting your private data and your e-commerce transactions is a mathematical hard problem that is only “hard” until someone comes a long and finds a way around it.
On January 7th 2010, a group posted a white paper[1] discussing in detail how they have factored and effectively broken 768 bit RSA encryption. Now, there is no need to go jump off the Golden Gate bridge yet, most all of your banking transactions are currently using 1024 bit RSA or 2048 bit RSA keys. But understand that mathematical hard problems are only “hard” problems until someone finds a neat way of making it no longer a hard problem. If you would like some light reading, the white paper can be found here: http://eprint.iacr.org/2010/006.pdf
So the advice I would give is be careful and always use the latest encryption methods available. Further never trust people that will make blanket statements to the effect that you have nothing to worry about because it will take billions of years to crack the encryption and that the Sun will have already gone super nova.
[1] Factorization of a 768-bit RSA modulus version 1.0, January 7, 2010, http://eprint.iacr.org/2010/006.pdf
When we peel back the layers of the onion, modern encryption is all about mathematical hard problems. Meaning, that the mathematical functions are easy to process one way and very difficult to process in the reverse without a specific piece of information. Take for example the RSA algorithm that has been protecting internet e-commerce for years and is the common algorithm used for the creation of nearly all digital certificates. At the very foundation, RSA is about dealing with two very large prime numbers. These two large prime numbers are multiplied together to create a new number. This new number is then used in the algorithm to encrypted your data. Now the mathematical hard problem is that given the new number it is very difficult to factor that new number down to the two original prime numbers that created it. Fundamentally all that is protecting your private data and your e-commerce transactions is a mathematical hard problem that is only “hard” until someone comes a long and finds a way around it.
On January 7th 2010, a group posted a white paper[1] discussing in detail how they have factored and effectively broken 768 bit RSA encryption. Now, there is no need to go jump off the Golden Gate bridge yet, most all of your banking transactions are currently using 1024 bit RSA or 2048 bit RSA keys. But understand that mathematical hard problems are only “hard” problems until someone finds a neat way of making it no longer a hard problem. If you would like some light reading, the white paper can be found here: http://eprint.iacr.org/2010/006.pdf
So the advice I would give is be careful and always use the latest encryption methods available. Further never trust people that will make blanket statements to the effect that you have nothing to worry about because it will take billions of years to crack the encryption and that the Sun will have already gone super nova.
[1] Factorization of a 768-bit RSA modulus version 1.0, January 7, 2010, http://eprint.iacr.org/2010/006.pdf
Subscribe to:
Posts (Atom)