I saw this article by David Sparks recently. As a fellow hobbyist who has been trying to improve my skills in programming for a couple of years, I've developed some strong opinions on how to start.
First an aside for my aspiring programmer hobbyist friends. One of the pieces of advice that I often see is that you simply need to have something you want to build and its all down hill from there. I found this advice to be demotivating (I recognize that the intent of the advice is not meant as demotivating but the impact was demotivating for me). I wanted to build things at a level well above what I could understand. Was I simply not cut out to understand Object Oriented Programming? Was Objective C beyond me? etc. That said, the advice is true. However, there are a few starting pieces that experienced programmers don't really have the right perspective to answer. There are some basics you need to put in place to make some progress.
The way I learn, may not work for you so I do suggest you do use a couple of different ways to learn to find out what works best for you. That said, I'd start with an organized course. My stance boils down to this. Go for good teachers.
Start with an organized course. Rather than searching out courses on the web, start with Coursera. If you are starting right at the beginning, I recommdend Programming for Everyone (Python). If you have some experience (know for loops, if / then, etc. then I recommend tackling the series from Rice University beginning with An Introduction to Interactive Programming in Python (Part 1).
Yes I recommended a language. Yes it is Python. No it is not Swift. When the Apple recommended Swift content is adopted, I would absolutely recommend it. At the moment, you have some experienced University professors teaching a basic level for free in a simple language that is widely used including this guy. The courses are broken up into doable sized chunks of as short as 4 weeks. These courses also have peer assisted or auto-graded assignments and quizzes. Its simply amazing this stuff is available from talented teachers.
With honorable mention, I've learned a lot from Simon Allardice on Lynda.com. That includes getting started Swift. Also, there are good tutorials but they're hit or miss. If you find some you like, go to town on them.
Also, helpful but for later in my opinion is iTunes U and the Stanford courses including Programming Methodology. It is a bit old but I think conceptually very helpful. I made it through Programming Methodology but not yet through the next two courses. These are also taught by world class teachers. However, the subject matter is much denser and its not quite as easy to fit a college level course into a hobbyists life.
Organized courses are the best. Start there first.
Books are hit and mostly miss for me. I like the Big Nerd Ranch books for typing along and making things work in OS X and iOS. The issue with books for iOS or OS X is that the language moves quickly enough from year to year that the older books have enough differences that the novice me often can't make them work. Go with the videos first and then when sufficiently comfortable, go to books from BNR on the topic of interest. I am about 80% through the newest Cocoa Programming in Swift so I do recommend that. However, its already destined to be more challenging to work through as Swift 2 is adopted. Even though the BNR folks are working through how to update for Swift 2
Have fun. Follow your own path. Learn PHP or Perl if you want. That said, I do strongly suggest you look for the best teachers you can find.