Happy new year everybody!
Everyone knows Google, right? They have a huge data-center that stores all website around the world. How can they do it? With a bot spider! Very intelligent spider. And we gonna made it, but simpler, and for our own purpose.
We don't have to defeat Google or competitive with them, we just learn from them to create ours.
For my reason, I want to create a bot, a spider or a crawler that can help me collect all articles from some e-news websites, some blogs, and from Google caches (for website is dead). Cut out the content and put it in my database (for my website, for my other activities, etc).
I hope you come her with same purpose as me, or have the same attitude to learn how crawler work. I'm really want to share and learn about new thing with other people. And I'm always open for your contribute, your comment and feedback to improve other skill.
Feel free to join group, feel free to prove me wrong (both coding and English). And I'm welcome you to be co-founder of this group too.
P/S: I'm new to python, so I don't know much, but I will try to post 1 tutorial per week around topic crawler and scrapping.
A few worlds about myself:
I'm Thang, a web developer and designer (*I can use photoshop and html/css too*), and I'm 24 y/o (*so you can call me bro, dude, noob if you want*). I used to work as freelancer, now I'm build e-com website for my friend. And I will find a new job on June, this year.
Haha no way... My very first SETT post was about writing a scraper in perl and using wGet. Small world, although it's probably more likely that the people who read Tynan's blog and thus are likely to use SETT are all very similar... Data mining
So back in January, I wrote out my 7 goals for the year. It's been two months, so let's see how I'm doing :
1. Become FULLY polyphasic
I'm close on this one. Many days I go perfectly, sometimes if I have nothing to do I oversleep and then skip some naps during the day. I'm actually pretty satisfied with that, as I'm only sleeping 2.5-4.5 hours per night, I'm never tired, and can always count on being awake early and staying up late. I'll keep pressing to be more consistent, but I'm satisfied with where I am.
I don't believe in New Years resolutions or any of that mumbojumbo, arbitrary dates. Frankly I hate New Years parties; they are loud, obnoxious and really pointless. Anyways, one of my favorite bloggers, Sean Ogle, recently wrote a post about what he did right and what he did wrong in the past year as a means of looking back.
I decided I should do the same thing. And thus, without further ado, what I think I did right in and the things I did 2012.
1. I ate a ton of good food. This past year I had some of the most amazing food experiences I can think of. I had amazing seafood in Madrid, out of this world innovative cuisine at countless New American/Gastropubbish restaurants in NYC and explored various high-end farmers markets and pastry shops. Furthermore, this year I managed to get tons of my friends into the knack of finding and appreciating good food. Overall this past year has been one of the most amazing years I've ever had for food, and I doubt I will be able to repeat it since I now live far away from NYC. Regardless, my respect of high-quality food is an all time high it is one of those things that I think having everyday makes one very happy.
2. I wrote a lot more and got more serious. For the majority of last year I was in a half daze, I played video games, Spent a lot of time redditing or doing stupid shit with my time. Of course I would still read, practice some languages or occasionally update my blog, but I wasn't serious enough. Then around august or September I decided to get serious. It finally clicked for me that Video games where being a huge waste of my time, and that honestly, I had already had my fair share. I also realized that social distractions like reddit, facebook, constant texting/messaging and reacting to everything people said (if you live in a house with others) was being a detriment to my success. Thus I significantly cut down my time on reddit and reacting to everything; facebook and texting still remained constant until around late November and at the moment I literally only check for notifications. I started writing a lot more just to get in the habit of it. I also started researching how I could make money online and read almost twice as many books in 3 months as I did the past 9 months.