Happy new year everybody!
Everyone knows Google, right? They have a huge data-center that stores all website around the world. How can they do it? With a bot spider! Very intelligent spider. And we gonna made it, but simpler, and for our own purpose.
We don't have to defeat Google or competitive with them, we just learn from them to create ours.
For my reason, I want to create a bot, a spider or a crawler that can help me collect all articles from some e-news websites, some blogs, and from Google caches (for website is dead). Cut out the content and put it in my database (for my website, for my other activities, etc).
I hope you come her with same purpose as me, or have the same attitude to learn how crawler work. I'm really want to share and learn about new thing with other people. And I'm always open for your contribute, your comment and feedback to improve other skill.
Feel free to join group, feel free to prove me wrong (both coding and English). And I'm welcome you to be co-founder of this group too.
P/S: I'm new to python, so I don't know much, but I will try to post 1 tutorial per week around topic crawler and scrapping.
A few worlds about myself:
I'm Thang, a web developer and designer (*I can use photoshop and html/css too*), and I'm 24 y/o (*so you can call me bro, dude, noob if you want*). I used to work as freelancer, now I'm build e-com website for my friend. And I will find a new job on June, this year.
Haha no way... My very first SETT post was about writing a scraper in perl and using wGet. Small world, although it's probably more likely that the people who read Tynan's blog and thus are likely to use SETT are all very similar... Data mining
So back in January, I wrote out my 7 goals for the year. It's been two months, so let's see how I'm doing :
1. Become FULLY polyphasic
I'm close on this one. Many days I go perfectly, sometimes if I have nothing to do I oversleep and then skip some naps during the day. I'm actually pretty satisfied with that, as I'm only sleeping 2.5-4.5 hours per night, I'm never tired, and can always count on being awake early and staying up late. I'll keep pressing to be more consistent, but I'm satisfied with where I am.
The thinks you never thunk you'd think....
Well good morning everyone. I was having a conversation with a dear friend of mine and she said the silliest thing. She said and I am quoting here, “I'll have kids when I'm ready.” and I started to laugh because trust me... mine is 23 and I'm still not ready! You aren't EVER ready to learn all of the things kids can teach you. Am I talking spiritual, inspirational and lovely little cutesy things they say like, “If fairies light up can we make one a night light?” NO! I am talking real kids here!
You are never ready for all the crap that they teach you because this is info you don't ever think you'll need and then can't imagine why you do! These are things like how to explain why we don't put peanut butter on the toilet seat! Horrid things like how to get lollipops off of the dog. You start asking yourself ignorant questions. Crazy things you never thought you'd hear yourself think. Things like... Why in the hell don't they make a Barbie that flushes down properly? Sickening things like, “Why is it only the most expensive Ninja Turtle that destroys the garbage disposal? Are the others less Ninja? And if they are Ninja why does the plumber tell me he sees them every week?”
As a parent you are never prepared people! That's just what they tell you so that the race doesn't end! You will inevitably end up walking around mumbling the most ignorant questions that you really NEED the answer to. Queries such as, “Well I don't really know if my mouth would fit around a doorknob. Can I put it on a resume?” You might even google the place where cootie vaccinations are available. You will find yourselves just smacking yourself in the head trying to get kid's thought to take hold... in that place where you thought you had a brain but now... dark and cold.
People without children don't understand why we want a union! Take for example this scenario. I was sitting with my daughter when she was a toddler with all of my friends standing around. We were talking about how she always only asked me questions that I never had the answer to. The conversation went something like this: