webscraping – Circadian https://circadian-capital.com capturing intelligence Wed, 17 Jun 2015 15:56:46 +0000 en-US hourly 1 https://wordpress.org/?v=5.6.17 146252634 The rich are different https://circadian-capital.com/rich-are-different/ https://circadian-capital.com/rich-are-different/#respond Tue, 17 Mar 2015 12:06:56 +0000 http://www.circadian-capital.co.uk/?p=2507 As part of an exploratory dialogue someone from our old world (longevity) asked if we might use some data intelligence to tease out a relationship. The relationship in question was individual wealth versus size of life insurance policy. The relationship mattered because an investor holding longevity risk will see her returns suffer (substantially) as the […]

The post The rich are different appeared first on Circadian.

]]> https://circadian-capital.com/rich-are-different/feed/ 0 2507 Harvesting medical data: a small project https://circadian-capital.com/harvesting-medical-data-a-small-project/ https://circadian-capital.com/harvesting-medical-data-a-small-project/#respond Wed, 21 Jan 2015 12:37:46 +0000 http://www.circadian-capital.co.uk/?p=2394 Going through some old files I came across this piece of work we performed for a client some time ago - Search Project “…specific/sensitive biomarkers in certain cancers from the scientific literature and other online sources. For example might we be able to search in pancreatic or lung cancer for reports of biomarkers with say […]

The post Harvesting medical data: a small project appeared first on Circadian.

]]> https://circadian-capital.com/harvesting-medical-data-a-small-project/feed/ 0 2394 PDF is evil! https://circadian-capital.com/pdf-is-evil/ https://circadian-capital.com/pdf-is-evil/#respond Sun, 09 Nov 2014 17:12:58 +0000 http://www.circadian-capital.co.uk/?p=1460 Hopefully you’re already convinced that you can grab whatever data you wish from webpages (and if not, later examples will hopefully tip the balance in our favour) but sooner or later you are going to ask about pdf. And let’s face it, when the latest report comes out that everyone wants to be able to […]

The post PDF is evil! appeared first on Circadian.

]]> https://circadian-capital.com/pdf-is-evil/feed/ 0 1460 Combined Harvester https://circadian-capital.com/combined-harvester/ https://circadian-capital.com/combined-harvester/#respond Sun, 19 Oct 2014 14:26:03 +0000 http://jwpress.clients.origin-partners.com/?p=1001 Let’s change gears, skip the code, and look at a harvester in action. Here’s the url for you to play with and a user-guide / description – Data Harvesting: Stock Prices From the S&P500 we choose up to 5 tickers for their daily adjusted-closes. Store those into memory with “Load Prices” against a time-frame of our […]

The post Combined Harvester appeared first on Circadian.

]]> https://circadian-capital.com/combined-harvester/feed/ 0 1001 101 Data Harvesting https://circadian-capital.com/101-data-scrape/ https://circadian-capital.com/101-data-scrape/#respond Thu, 18 Sep 2014 14:31:19 +0000 http://jwpress.clients.origin-partners.com/?p=919 Building out a fully-fledged data-harvesting bot is not our purpose here, instead we will quickly demonstrate a 101 harvester in action against a financial variable. Even though simplified, the exercise is useful: the modules of work shown below are consistent for any level of complexity. Let’s work with FX since it is straightforward. (We’ll likely look at […]

The post 101 Data Harvesting appeared first on Circadian.

]]> https://circadian-capital.com/101-data-scrape/feed/ 0 919 A stupefying evening, one dead mouse and a python… https://circadian-capital.com/a-stupefying-evening-one-dead-mouse-and-a-python/ https://circadian-capital.com/a-stupefying-evening-one-dead-mouse-and-a-python/#respond Sat, 09 Aug 2014 19:22:30 +0000 http://jwpress.clients.origin-partners.com/?p=440 Late one evening reading Nathan Yau’s excellent “Visualize This” some years ago, I stumbled upon this undecipherable – url = 'http://www.wunderground.com/history/airport/KBUF/2009/1/1/DailyHistory.html' page = urllib2.urlopen(url) soup = BeautifulSoup(page) dayTemp = soup.findAll('span')[4].text print dayTemp Apparently there were Pythons involved, the soup was a parser, and we now knew the weather in Buffalo for 1 Jan 2009. Of […]

The post A stupefying evening, one dead mouse and a python… appeared first on Circadian.

]]> https://circadian-capital.com/a-stupefying-evening-one-dead-mouse-and-a-python/feed/ 0 440