pywind.ofgem

pywind.ofgem.objects

class pywind.ofgem.objects.CertificateStation(name, g_id, capacity, scheme)

Bases: object

We are normally interested in knowing about certificates issued to a station, so this class attempts to simplify this process. Once issued all certificates will be accounted for, but the final owner and status may change. This class attempts to take a bunch of Certificate objects and simplify them into a final set, with ownership and status correctly attributed.

add_cert(cert)

as_row()

class pywind.ofgem.objects.Certificates(node)

Bases: pywind.ofgem.objects.OfgemObjectBase

Certificate Number Fact Sheet https://www.ofgem.gov.uk/sites/default/files/docs/roc_identifier_fact_sheet_dec_2015.pdf

XML_MAPPING = (('textbox4', 'generator_id'), ('textbox13', 'name'), ('textbox5', 'scheme'), ('textbox19', 'capacity', 'float', 0.0), ('textbox12', 'country'), ('textbox15', 'technology'), ('textbox31', 'generation_type'), ('textbox18', 'period'), ('textbox21', 'certs', 'int', 0), ('textbox24', 'start_no'), ('textbox27', 'finish_no'), ('textbox37', 'factor', 'float', 0.0), ('textbox30', 'issue_dt', 'date'), ('textbox33', 'status'), ('textbox36', 'status_dt', 'date'), ('textbox39', 'current_holder'), ('textbox45', 'reg_no'))

certificates

Number of certificates covered by this object.

Return type:int

digits

Number of digits that store the certificate number.

Return type:int

finish

Return the numeric finish number for the certificates. Each certificate number contains the station, period and the number of the certificate, so this function extracts the numeric part.

Returns:Finish number of the certificates referenced Return type:integer

output

Calculate the output based on the number of certs issued and factor.

Returns:Numeric output or 0 Return type:float

output_summary()

Return a string with the output for the certificates.

Return type:str

start

Return the numeric start number for the certificates. Each certificate number contains the station, period and the number of the certificate, so this function extracts the numeric part.

Returns:Start number of the certificates referenced Return type:int

station_details()

Get a dict object with the station information for these certificates.

Returns:Dict with just information relevant to identifying the station Return type:dict

class pywind.ofgem.objects.OfgemObjectBase(node)

Bases: object

XML_MAPPING = None

as_row()

Return the information in correct format for rows() usage

Returns:Formatted attribute dict Return type:dict

class pywind.ofgem.objects.Station(node)

Bases: pywind.ofgem.objects.OfgemObjectBase

Store details of a single station using data from Ofgem.

The exposed object makes the individual pieces of data available by acting as a dict, i.e. .. :code:

name = station['name']

The convenience function as_string() will return a full list of the data formatted for display in a terminal.

XML_MAPPING = (('GeneratorID', 'generator_id'), ('StatusName', 'status'), ('GeneratorName', 'name'), ('SchemeName', 'scheme'), ('Capacity', '', 'float'), ('Country',), ('TechnologyName', 'technology'), ('OutputType', 'output'), ('AccreditationDate', 'accreditation_dt', 'date'), ('CommissionDate', 'commission_dt', 'date'), ('textbox6', 'developer'), ('textbox61', 'developer_address', 'address'), ('textbox65', 'address', 'address'), ('FaxNumber', 'fax'))

pywind.ofgem.search

class pywind.ofgem.search.CertificateSearch(filename=None)

Bases: object

Getting information about certificates issued by Ofgem requires accessing their webform. This class provides a simple way of doing that. Class that queries ofgem for certificate data. If it succeeds then

There are 2 generator methods that allow iterating through the returned data, - each call to stations() will return a list of Certificates objects related to a single station. - each call to certificates() will return a single Certificates object.

>>> from pywind.ofgem.CertificateSearch import CertificateSearch
>>> ocs = CertificateSearch()
>>> ocs.start()
True
>>> ocs.set_period(201601)
True
>>> ocs.get_data()
True
>>> len(ocs)
4898

NSMAP = {'a': 'CertificatesExternalPublicDataWarehouse'}

START_URL = 'ReportViewer.aspx?ReportPath=/DatawarehouseReports/CertificatesExternalPublicDataWarehouse&ReportVisibility=1&ReportCategory=2'

certificates()

Generator that returns Certificates objects.

Returns:Certificates objects Return type:Certificates

filter_generation_type(what)

Filter certificates by generation type

filter_generator_id(acc_no)

Filter certificates by generator id (accreditation number).

Note

Values supplied are upper cased automatically.

Parameters:acc_no – Accreditation/Generation number Return type:bool

filter_scheme(what)

Filter certificates by scheme

Parameters:what – Scheme abbreviation [REGO, RO] Return type:bool

filter_technology(what)

Filter certificates by technology group

get_data()

Submit the form, get the results and parse them into Certificate objects

Return type:bool

parse_filename(filename)

Parse an Ofgem generated file of certificates. This parses downloaded Ofgem files.

Parameters:filename – The filename to be parsed Returns:True or False Return type:bool

rows()

Generator function that returns a station each time it is called.

Returns:A function that returns a dict containing information on one station. Return type:generator

save_original(filename)

Save the downloaded certificate data into the filename provided.

Parameters:filename – Filename to save the file to. Return type:bool

set_finish_month(month)

Set the finish month for certificates

Parameters:month – Numeric month number Return type:bool

set_finish_year(year)

Set the finish year for certificates

Parameters:year – Numeric year to be set Return type:bool

set_period(yearmonth)

Set the year and month for certificates.

Parameters:yearmonth – Numeric period in YYYYMM format Returns:True or False Return type:bool

set_start_month(month)

Set the start month for certificates

Parameters:month – Numeric month number Return type:bool

set_start_year(year)

Set the start year for certificates

Parameters:year – Numeric year to be set Return type:bool

start()

Retrieve the form from Ofgem website so we can start updating it.

Returns:True or False Return type:bool

stations()

Generator that returns a Return a list of stations related to the certificates

class pywind.ofgem.search.StationSearch

Bases: object

Performing a station search using the Ofgem website takes a while due to the 3.5M initial file and the 2M replies that are sent. Parsing these takes time, so patience is needed.

>>> from pywind.ofgem.StationSearch import StationSearch
>>> oss = StationSearch()
>>> oss.start()
True
>>> oss.filter_name('griffin')
True
>>> oss.get_data()
True
>>> len(oss)
4

START_URL = 'ReportViewer.aspx?ReportPath=/Renewables/Accreditation/AccreditedStationsExternalPublic&ReportVisibility=1&ReportCategory=1'

filter_generator_id(accno)

Filter stations based on generator id.

Return type:bool

filter_name(name)

Filter stations based on name. The search will return all stations containing the supplied name.

Parameters:name – The name to filter for Return type:bool

filter_organisation(org_name)

Filter stations based on generator id.

Parameters:org_name – Organisation name to filter Return type:bool

filter_scheme(scheme)

Filter stations based on scheme they are members of.

Return type:bool

filter_technology(what)

Filter stations based on technology.

Return type:bool

get_data()

Get data from form.

Return type:bool

rows()

Generator to return dicts of station information.

Returns:Dict of station information Return type:dict

save_original(filename)

Save the downloaded station data into the filename provided.

Parameters:filename – Filename to save the file to. Return type:bool

start()

Retrieve the form from Ofgem website so we can start updating it.

pywind.ofgem.form

To get data from the Ofgem website we need to use one of their web forms. These use MS javascript to work and were originally only usable in IE 8 or IE9. Modern versions are usable in more browsers but the forms themselves have also evolved and are more complex than is ideal.

class pywind.ofgem.form.OfgemForm(url)

Bases: object

Class to represent an instance of an Ofgem form.

get()

Attempt to get the initial version of the form from the website.

save_original(filename)

Save the original, downloaded source into the filename provided.

Parameters:filename – Filename to save the file to. Returns:True or False Return type:boolean

set_value(lbl, value)

submit()

Submit the form data and update based on response. Given how slow the parsing of a 3M HTML page is, try and use the X-MicrosoftAjax: Delta=true header to get smaller blocks for processing.

update()

Submit the form data and update based on response. Given how slow the parsing of a 3M HTML page is, try and use the X-MicrosoftAjax: Delta=true header to get smaller blocks for processing.

pywind.ofgem.form_data

Each Ofgem web form contains a lot of information. Classes in this file try to make managing the data easier.

class pywind.ofgem.form_data.FormData(initial_data='', stored_file=None)

Bases: object

Class to store and allow easy manipulation of data from an Ofgem form.

as_post_data(quoted=True, submit=False)

Process the form elements and return in a dict suitable for using as POST data.

Parameters:

• quoted – If set the returned data will be fully quoted.
• submit – True only if this is a submission post.

Returns:

Dict of data to be posted as name: value pairs

Return type:

dict

set_value_by_label(lbl, value)

Set a value based on a label.

update(content='')

Given some content, update the form.

Parameters:content – The content to update the form from Returns:True or False Return type:bool

value_for_label(lbl)

pywind.ofgem.form_data.element_attributes(elm)

Return a dict of the basic attributes we want from an XML element.

pywind.ofgem.form_data.quote('abc def') → 'abc%20def'

Each part of a URL, e.g. the path info, the query, etc., has a different set of reserved characters that must be quoted.

RFC 2396 Uniform Resource Identifiers (URI): Generic Syntax lists the following reserved characters.

reserved = ”;” | “/” | ”?” | ”:” | “@” | “&” | “=” | “+” |

“$” | ”,”

Each of these characters is reserved in some component of a URL, but not necessarily in all of them.

By default, the quote function is intended for quoting the path section of a URL. Thus, it will not encode ‘/’. This character is reserved, but in typical usage the quote function is being called on a path where the existing slash characters are used as reserved characters.

pywind.ofgem.form_data.selected_list(element)

Given an element dict, return a list of the indexes.