aboutsummaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
authorMatěj Cepl <mcepl@cepl.eu>2024-01-19 14:47:11 +0100
committerMatěj Cepl <mcepl@cepl.eu>2024-01-19 14:47:47 +0100
commitd841f17857475f6024f020123d7dafff3bd6e8ee (patch)
tree00a2c7551e432a07810c412a13b8b9f7d3ddd6a2 /README.md
parent63d11b17b4e9cd8c33308b50f56c0af2e1b0583c (diff)
downloadgg_scraper-master.tar.gz
Switch README to Markdown and add LICENSE.HEADmaster
Diffstat (limited to 'README.md')
-rw-r--r--README.md49
1 files changed, 49 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..54d636d
--- /dev/null
+++ b/README.md
@@ -0,0 +1,49 @@
+# Google Group Scrapper
+
+![Build Status](https://secure.travis-ci.org/mcepl/gg_scraper.png)
+
+A small script as a replacement of [the old PHP
+script](http://saturnboy.com/2010/03/scraping-google-groups/) for
+downloading messages stored in the black hole of the Google Groups.
+
+## How to use it?
+
+This script requires `formail(1)` from `procmail` package. Any version
+is OK, so please install it from your distribution's repositories. Then
+run:
+
+ pip install beautifulsoup4 PyYAML
+ python gg_scraper.py 'https://groups.google.com/forum/#!forum/<group_name>'
+
+## Background
+
+I would never start without an inspiration from the
+[comment](https://luther.ceplovi.cz/blog/2013/09/19/we-should-stop-even-pretending-google-is-trying-to-do-the-right-thingtm/#comment-133-by-sean-hogan)
+by Sean Hogan on my previous post on the theme of locked down nature of
+Google Groups:
+
+> At least google-groups appears to follow google\'s own advice on
+> making AJAX sites accessible to the google web-crawler. See
+> <https://developers.google.com/webmasters/ajax-crawling/docs/getting-started>
+>
+> So for me,
+> <http://groups.google.com/d/forum/jbrout:FBv1oMXRZkxB6YShaFuNHc3-Moc&cuid=3654582>
+> redirects (eventually) to
+> <https://groups.google.com/forum/#!forum/jbrout> which can be viewed
+> in raw HTML as
+> <https://groups.google.com/forum/?_escaped_fragment_=forum/jbrout>
+>
+> Google Groups seems a perfect use-case for
+> extreme-progressive-enhancement, but what would I know.
+>
+> regards, Sean
+
+------------------------------------------------------------------------
+
+All issues, questions, complaints, or (even
+better!) patches should be send via email to
+[~mcepl/<devel@lists.sr.ht>](mailto:~mcepl/devel@lists.sr.ht)
+email list (for patches use [git
+send-email](https://git-send-email.io/)). For the issue tracking
+I use [git-bug](https://github.com/MichaelMure/git-bug) in this
+repo.