

{"id":1730,"date":"2012-10-24T06:12:16","date_gmt":"2012-10-24T11:12:16","guid":{"rendered":"http:\/\/rud.is\/b\/?p=1730"},"modified":"2020-02-24T16:24:43","modified_gmt":"2020-02-24T21:24:43","slug":"extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv","status":"publish","type":"post","link":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/","title":{"rendered":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python"},"content":{"rendered":"<p>I played around with <a href=\"https:\/\/wordpress.org\/plugins\/ose-firewall\/\">OSE Firewall<\/a> for WordPress for a couple days to see if it was worth switching to from the plugin I was previously using. It&#8217;s definitely not as full featured and I didn&#8217;t see any WP database extensions where it kept a log I could review\/analyze, so I whipped up a little script to extract all the alert data from the Gmail account I setup for it to log to.<\/p>\n<p>The script below \u2013\u00a0while focused on getting OSE Firewall alert data \u2013\u00a0can be easily modified to search for other types of automated\/formatted e-mails and build a CSV file with the results. Remember, tho, that you&#8217;re going to be putting your e-mail credentials in this file (if you end up using it) so either use a mailbox you don&#8217;t care about or make sure you use sane permissions on the script and keep it somewhere safe.<\/p>\n<p>I tested it on linux boxes, but it should work anywhere you have Python and mailbox access.<\/p>\n<p>I highly doubt there will be any updates to this version (I&#8217;m not using OSE Firewall anymore), but you an grab the source below or <span class=\"removed_link\" title=\"https:\/\/github.com\/hrbrmstr\/osefw\">on github<\/span>. There should be sufficient annotation in the comments, but if you have any questions, drop a note in the comments.<\/p>\n<pre><code class=\"language-python\"># oswfw.py - extract WordPress OSE Firewall mail alerts to CSV\n# \n# Author: @hrbrmstr\n#\n\nimport imaplib\nimport datetime\nimport re\n\n# get 'today' (in the event you are just reporting on today's hits\ndate = (datetime.date.today() - datetime.timedelta(1)).strftime(\"%d-%b-%Y\")\n\n# setup IMAP connection\n\ngmail = imaplib.IMAP4_SSL('imap.gmail.com',993) # use your IMAP server it not Gmail\ngmail.login(\"YOUR_IMAP_USERNAME\",\"YOUR_PASSWORD\")\ngmail.select('[Gmail]\/All Mail') # Your IMAP's \"all mail\" if not using Gmail\n\n# now search for all mails with \"OSE Firewall\" in the subject\n\n# uncomment this line and comment out the next one to just get results from 'today'\n#result, data = gmail.uid('search', None, '(SENTSINCE {date} HEADER Subject \"OSE Firewall*\")'.format(date=date))\nresult, data = gmail.uid('search', None, '(HEADER Subject \"OSE Firewall*\")')\n\n# setup CSV file for output\n\nf = open(\"osefw.csv\", \"w+\")\nf.write(\"Date,IP,URI,Method,UserAgent,Referer\\n\") ;\n\n# cycle through result set from IMAP search query, extracting salient info\n# from headers\/body of each found message\n\nfor msg in data[0].split():\n\n    # fetch the msg for the UID\n    res, msg_txt = gmail.uid('fetch', msg, '(RFC822)')\n\n    # get rid of carriage returns\n    body = re.sub(re.compile('\\r', re.MULTILINE), '', msg_txt[0][1])\n\n    # extract salient fields from the message body\/header\n    DATE = re.findall('^Date: (.*?)$', body, re.M)\n    IP = re.findall('^FROM IP: http:\\\/\\\/whois.domaintools.com\\\/(.*?)$', body, re.M)\n    URI = re.findall('^URI: (.*?)$', body, re.M)\n    METHOD = re.findall('^METHOD: (.*?)$', body, re.M)\n    USERAGENT= re.findall('^USERAGENT: (.*?)$', body, re.M)\n    REFERER = re.findall('^REFERER: (.*?)$', body, re.M)\n\n    # format for CSV output\n    ose_log  = \"%s,%s,%s,%s,%s,%s\\n\" % (DATE, IP, URI, METHOD, USERAGENT, REFERER)\n\n    # quicker to replace array output brackets than to deal with non-array results checking\n    f.write(re.sub(\"[\\[\\]]*\", \"\", ose_log))\n\n    f.flush() ;\n\ngmail.logout()\nf.close()\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>I played around with OSE Firewall for WordPress for a couple days to see if it was worth switching to from the plugin I was previously using. It&#8217;s definitely not as full featured and I didn&#8217;t see any WP database extensions where it kept a log I could review\/analyze, so I whipped up a little [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":true,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"activitypub_content_warning":"","activitypub_content_visibility":"","activitypub_max_image_attachments":3,"activitypub_interaction_policy_quote":"anyone","activitypub_status":"","footnotes":""},"categories":[63,3,640],"tags":[],"class_list":["post-1730","post","type-post","status-publish","format-standard","hentry","category-development","category-information-security","category-python-2"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.2 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is\" \/>\n<meta property=\"og:description\" content=\"I played around with OSE Firewall for WordPress for a couple days to see if it was worth switching to from the plugin I was previously using. It&#8217;s definitely not as full featured and I didn&#8217;t see any WP database extensions where it kept a log I could review\/analyze, so I whipped up a little [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\" \/>\n<meta property=\"og:site_name\" content=\"rud.is\" \/>\n<meta property=\"article:published_time\" content=\"2012-10-24T11:12:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-02-24T21:24:43+00:00\" \/>\n<meta name=\"author\" content=\"hrbrmstr\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"hrbrmstr\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\"},\"author\":{\"name\":\"hrbrmstr\",\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"headline\":\"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python\",\"datePublished\":\"2012-10-24T11:12:16+00:00\",\"dateModified\":\"2020-02-24T21:24:43+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\"},\"wordCount\":232,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"articleSection\":[\"Development\",\"Information Security\",\"Python\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\",\"url\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\",\"name\":\"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is\",\"isPartOf\":{\"@id\":\"https:\/\/rud.is\/b\/#website\"},\"datePublished\":\"2012-10-24T11:12:16+00:00\",\"dateModified\":\"2020-02-24T21:24:43+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/rud.is\/b\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/rud.is\/b\/#website\",\"url\":\"https:\/\/rud.is\/b\/\",\"name\":\"rud.is\",\"description\":\"&quot;In God we trust. All others must bring data&quot;\",\"publisher\":{\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/rud.is\/b\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886\",\"name\":\"hrbrmstr\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"url\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\",\"width\":460,\"height\":460,\"caption\":\"hrbrmstr\"},\"logo\":{\"@id\":\"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1\"},\"description\":\"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7\",\"sameAs\":[\"http:\/\/rud.is\"],\"url\":\"https:\/\/rud.is\/b\/author\/hrbrmstr\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/","og_locale":"en_US","og_type":"article","og_title":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is","og_description":"I played around with OSE Firewall for WordPress for a couple days to see if it was worth switching to from the plugin I was previously using. It&#8217;s definitely not as full featured and I didn&#8217;t see any WP database extensions where it kept a log I could review\/analyze, so I whipped up a little [&hellip;]","og_url":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/","og_site_name":"rud.is","article_published_time":"2012-10-24T11:12:16+00:00","article_modified_time":"2020-02-24T21:24:43+00:00","author":"hrbrmstr","twitter_card":"summary_large_image","twitter_misc":{"Written by":"hrbrmstr","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#article","isPartOf":{"@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/"},"author":{"name":"hrbrmstr","@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"headline":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python","datePublished":"2012-10-24T11:12:16+00:00","dateModified":"2020-02-24T21:24:43+00:00","mainEntityOfPage":{"@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/"},"wordCount":232,"commentCount":0,"publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"articleSection":["Development","Information Security","Python"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/","url":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/","name":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python - rud.is","isPartOf":{"@id":"https:\/\/rud.is\/b\/#website"},"datePublished":"2012-10-24T11:12:16+00:00","dateModified":"2020-02-24T21:24:43+00:00","breadcrumb":{"@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/rud.is\/b\/2012\/10\/24\/extracting-ose-firewall-alert-data-from-imap-gmail-mail-to-csv\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/rud.is\/b\/"},{"@type":"ListItem","position":2,"name":"Extracting OSE Firewall Alert Data From IMAP (Gmail) Mail To CSV With Python"}]},{"@type":"WebSite","@id":"https:\/\/rud.is\/b\/#website","url":"https:\/\/rud.is\/b\/","name":"rud.is","description":"&quot;In God we trust. All others must bring data&quot;","publisher":{"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/rud.is\/b\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/rud.is\/b\/#\/schema\/person\/d7cb7487ab0527447f7fda5c423ff886","name":"hrbrmstr","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","url":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","contentUrl":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1","width":460,"height":460,"caption":"hrbrmstr"},"logo":{"@id":"https:\/\/i0.wp.com\/rud.is\/b\/wp-content\/uploads\/2023\/10\/ukr-shield.png?fit=460%2C460&ssl=1"},"description":"Don't look at me\u2026I do what he does \u2014 just slower. #rstats avuncular \u2022 ?Resistance Fighter \u2022 Cook \u2022 Christian \u2022 [Master] Chef des Donn\u00e9es de S\u00e9curit\u00e9 @ @rapid7","sameAs":["http:\/\/rud.is"],"url":"https:\/\/rud.is\/b\/author\/hrbrmstr\/"}]}},"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p23idr-rU","jetpack_likes_enabled":true,"jetpack-related-posts":[{"id":1089,"url":"https:\/\/rud.is\/b\/2012\/05\/28\/slopegraphs-in-python\/","url_meta":{"origin":1730,"position":0},"title":"Slopegraphs in Python","author":"hrbrmstr","date":"2012-05-28","format":false,"excerpt":"(NOTE: You can keep up with progress best at github, but can always search on \"slopegraph\" here or just hit the tag page: \"slopegraph\" regularly) I've been a bit obsessed with slopegraphs (a.k.a \"Tufte table-chart\") of late and very dissatisfied with the lack of tools to make this particular visualization\u2026","rel":"","context":"In &quot;Charts &amp; Graphs&quot;","block_context":{"text":"Charts &amp; Graphs","link":"https:\/\/rud.is\/b\/category\/charts-graphs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":2618,"url":"https:\/\/rud.is\/b\/2013\/09\/04\/nizdos\/","url_meta":{"origin":1730,"position":1},"title":"nizdos &#8211; Nest Thermometer Notifications And Data Logging In Python","author":"hrbrmstr","date":"2013-09-04","format":false,"excerpt":"I've had a Nest thermometer for a while now and it's been an overall positive experience. It's given me more visibility into our heating\/cooling system usage, patterns and preferences; plus, it actually saved us money last winter. We try to avoid running the A\/C during the summer, and it would\u2026","rel":"","context":"In &quot;Development&quot;","block_context":{"text":"Development","link":"https:\/\/rud.is\/b\/category\/development\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1145,"url":"https:\/\/rud.is\/b\/2012\/06\/02\/slopegraphs-in-python-the-great-refactor\/","url_meta":{"origin":1730,"position":2},"title":"Slopegraphs in Python \u2013 The Great Refactor","author":"hrbrmstr","date":"2012-06-02","format":false,"excerpt":"Despite being on holiday, I had a spare hour to refactor the code (@mrshrbrmstr was joining the 1% in the hotel spa). It's up on github and now sports a spiffy JSON-format config file. You now must execute the slopegraph.py script with a \"--config FILENAME\" argument. The configuration file lets\u2026","rel":"","context":"In &quot;Charts &amp; Graphs&quot;","block_context":{"text":"Charts &amp; Graphs","link":"https:\/\/rud.is\/b\/category\/charts-graphs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3022,"url":"https:\/\/rud.is\/b\/2014\/09\/14\/an-alfred-os-x-workflow-for-weather-underground-personal-weather-station-pws-data\/","url_meta":{"origin":1730,"position":3},"title":"An Alfred (OS X) Workflow for Weather Underground Personal Weather Station (PWS) Data","author":"hrbrmstr","date":"2014-09-14","format":false,"excerpt":"I've operated a [Weather Underground](http:\/\/www.wunderground.com\/) [Personal Weather Station](http:\/\/www.wunderground.com\/weatherstation\/about.asp) (PWS) [[KMEBERWI7](http:\/\/www.wunderground.com\/personal-weather-station\/dashboard?ID=KMEBERWI7#history)] off-and-on (hardware issues notwithstanding) for as long as I can remember, and I thought it was about time to finally do an Alfred\u2194PWS mashup. My personal requirements were quite modest: - 5 reading history (including most current) - Ability to\u2026","rel":"","context":"In &quot;Alfred&quot;","block_context":{"text":"Alfred","link":"https:\/\/rud.is\/b\/category\/alfred\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3386,"url":"https:\/\/rud.is\/b\/2015\/05\/09\/quotebox-an-npr-like-embedded-twitter-quote-generator\/","url_meta":{"origin":1730,"position":4},"title":"quotebox &#8211; An NPR-like Embedded Twitter Quote Generator","author":"hrbrmstr","date":"2015-05-09","format":false,"excerpt":"I'm an avid NPR listener also follow a number of their programs and people on Twitter. I really dig their [quotable](https:\/\/github.com\/nprapps\/quotable) tweets. Here's a sample of a recent one: Minn. state senators cannot look other senators in the eye during floor debate. @ailsachang http:\/\/t.co\/SfQBq4yyHQ pic.twitter.com\/DNHGEiVA9j\u2014 NPR News (@nprnews) May 8,\u2026","rel":"","context":"In &quot;phantomjs&quot;","block_context":{"text":"phantomjs","link":"https:\/\/rud.is\/b\/category\/phantomjs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":1849,"url":"https:\/\/rud.is\/b\/2012\/12\/17\/easier-html-table-scraping-for-scripts-with-google-drive\/","url_meta":{"origin":1730,"position":5},"title":"Easier HTML Table-scraping For Scripts With Google Drive","author":"hrbrmstr","date":"2012-12-17","format":false,"excerpt":"We had our first, real, snowfall of the season in Maine today and that usually means school delays\/closings. Our \"local\" station \u2013 @WCHS6 \u2013 has a Storm Center Closings page as well as an SMS notification service. I decided this morning that I needed a command line version (and, eventually,\u2026","rel":"","context":"In &quot;Google Docs&quot;","block_context":{"text":"Google Docs","link":"https:\/\/rud.is\/b\/category\/google-docs\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/1730","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/comments?post=1730"}],"version-history":[{"count":0,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/posts\/1730\/revisions"}],"wp:attachment":[{"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/media?parent=1730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/categories?post=1730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rud.is\/b\/wp-json\/wp\/v2\/tags?post=1730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}