code: np.var(results, ddof=1)

rawDescription: How can I calculate the variance of a list in python?

description: Calculate the variance of a list in python


code:
import matplotlib.pyplot as plt
import matplotlib.dates as md
import numpy as np
import datetime as dt
import time

n=20
duration=1000
now=time.mktime(time.localtime())
timestamps=np.linspace(now,now+duration,n)
dates=[dt.datetime.fromtimestamp(ts) for ts in timestamps]
values=np.sin((timestamps-now)/duration*2*np.pi)
plt.subplots_adjust(bottom=0.2)
plt.xticks( rotation=25 )
ax=plt.gca()
xfmt = md.DateFormatter('%Y-%m-%d %H:%M:%S')
ax.xaxis.set_major_formatter(xfmt)
plt.plot(dates,values)
plt.show()

rawDescription: plotting unix timestamps in matplotlib

description: Plotting unix timestamps in matplotlib


rawDescription: How can I organize my data into a grid using Python Pandas?

description: Organize my data into a grid using Python Pandas


code:
hi_web = urllib2.urlopen('http://search.twitter.com/search.json?q=hi');
with open('hi.json', 'ab') as hi_file:
  hi_file.write(hi_web.read())

rawDescription: Save JSON outputed from a URL to a file

description: Save JSON outputed from a URL to a file


rawDescription: What is an efficient way to slice a Pandas Series with a MultiIndex along multiple dimensions?


code: h_size = len(h) * 4

rawDescription: Converting from hex to binary without losing leading 0's python


code:
def threads_over_lists(fn):
    def wrapped(x, *args, **kwargs):
        if isinstance(x, list):
            return [fn(e, *args, **kwargs) for e in x]
        return fn(x, *args, **kwargs)
    return wrapped

rawDescription: Automatically use list comprehension/map() recursion if a function is given a list


code:
def pyramid(h,symbol="#"):
    for i in range(h):
        print(" " * (h-i - 1) + symbol * (2 * i + 1))
    print()

pyramid(5)

rawDescription: Writing a function that prints a pyramid - two parameters


code:
rankdata(somelist)
len(somelist) - rankdata(somelist)
len(somelist) - rankdata(somelist) + 1

rawDescription: Ranking a List of Numbers

description: Ranking a List of Numbers


code: data

rawDescription: Divide dictionary values by another value in the same key in Python


code:
with open('/tmp/redirect.log', 'a') as log:
    while True: #If you need to continuously check for more.
        needed = {"GET", "User-Agent"}
        for line in sys.stdin:
            for item in needed:
                if line.startswith(item):
                    log.write(line)
                    break
            needed.remove(item)
            if not needed: #The set is empty, we have found all the lines we need.
                break

rawDescription: Python reading first four lines from readlines()


code:
A = [10, 20, 30, 40]

B = (20, 60, 81, 90)

from heapq import merge

for ele in merge(A,B):
    print(ele)

rawDescription: Better way for concatenating two sorted list of integers


code:
url_for('blog_post', id=post)
# /blog/1234/the-post-title

rawDescription: Create a url with an article's title

description: Create a url with an article's title


code:
import matplotlib.pyplot as plt
import numpy as np

def main():
    fig, ax = plt.subplots()
    ax.pcolormesh(np.random.random((100, 100)), cmap='gray')

    ClickToDrawPoints(ax).show()

class ClickToDrawPoints(object):
    def __init__(self, ax):
        self.ax = ax
        self.fig = ax.figure
        self.xy = []
        self.points = ax.scatter([], [], s=200, color='red', picker=20)
        self.fig.canvas.mpl_connect('button_press_event', self.on_click)

    def on_click(self, event):
        if event.inaxes is None:
            return
        self.xy.append([event.xdata, event.ydata])
        self.points.set_offsets(self.xy)
        self.ax.draw_artist(self.points)
        self.fig.canvas.blit(self.ax.bbox)

    def show(self):
        plt.show()

main()

rawDescription: Efficient Matplotlib Redrawing

description: Efficient Matplotlib Redrawing


code:
repo.index.commit('Merge master into feature',
parent_commits=(current.commit, master.commit))

rawDescription: How do you merge the master branch into a feature branch with GitPython?

description: Merge the master branch into a feature branch with GitPython


code:
with open("drag_and_drop_helper.js") as f:
    js = f.read()

driver.execute_script(js + "$('#one').simulateDragDrop({ dropTarget: '#bin'});")

rawDescription: How to simulate HTML5 Drag and Drop in Selenium Webdriver?

description: Simulate HTML5 Drag and Drop in Selenium Webdriver


code:
import itertools

mystring = 'abcde'
for i in range(1,len(mystring)+1):
    for combo in itertools.combinations(mystring, i):
        print(''.join(combo))

rawDescription: How to find the combinations (probability) for a,b,c,d,e

description: Find the combinations (probability) for a,b,c,d,e


code:
# Create user
username = 'TestUser'
email = 'test@example.com'
passw = '1234'
new_user = User.objects.create_user(username, email, passw)

# Create profile
phone = '654654654'
desc = 'Test user profile'
new_profile = Profile(user=new_user, phone = phone, description=desc)
new_profile.profile_role = new_u_prole
new_profile.user = new_user

# Save profile and user
new_profile.save()
new_user.save()

rawDescription: Django: Add another field to django-registration


code:
#!/usr/bin/python
# -*- coding:utf8 -*-

import pygame, sys

#init pygame
pygame.init()

#init screen
screen=pygame.display.set_mode((800,600))
screen.fill((255,0,255))

#loading the images
texture=pygame.image.load("texture.jpg").convert()
texture_rect=texture.get_rect()
texture_rect.center=(200,300)
mask=pygame.Surface((texture_rect.width,texture_rect.height)) # mask should have only 2 colors: black and white
mask.fill((255,255,255))
pygame.draw.circle(mask,(0,0,0),(texture_rect.width/2,texture_rect.height/2),int(texture_rect.width*0.3))
mask_rect=mask.get_rect()
mask_rect.center=(600,300)

tmp_image=texture.copy() # make a copy of the texture to keep it unchanged for future usage
mask.set_colorkey((0,0,0)) # we want the black colored parts of the mask to be transparent
tmp_image.blit(mask,(0,0)) # blit the mask to the texture. the black parts are transparent so we see the pixels of the texture there

tmp_rect=tmp_image.get_rect()
tmp_rect.center=(400,300)
tmp_image.set_colorkey((255,255,255))
screen.blit(texture,texture_rect)
screen.blit(mask,mask_rect)
screen.blit(tmp_image,tmp_rect)

pygame.display.flip()

while 1:
    event=pygame.event.wait()
    if event.type == pygame.QUIT or (event.type == pygame.KEYDOWN and event.key in [pygame.K_ESCAPE, pygame.K_q]):
        sys.exit()

rawDescription: Active texturing with pygame (possible? what concepts to look into?)


code:
xlApp = Excel.ExcelApp(False)
xlApp.convertTo2003('FileName.xlsx')

class ExcelApp(object):
    def __init__(self, visible):
        self.app = win32com.client.Dispatch('Excel.Application')
        if visible:
            self.app.Visible = True

    def __exit__(self):
        self.app.Quit()

    def __del__(self):
        self.app.Quit()

    def convertTo2003(self, fileName):
        if self.app:
            wb = self.app.WorkBooks.Open(fileName, ReadOnly = True)
            wb.SaveAs(fileName[:-1], FileFormat = 56)
            wb.Close()

    def quit(self):
        if self.app:
            self.app.Quit()

rawDescription: Write excel 2003 files from python 3.x

description: Write excel 2003 files from python 3.x


code:
df = 'path/to/data/file'
blocksize = 8192 # or some other size packet you want to transmit.
                 # Powers of 2 are good.
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

rawDescription: How to transfer a file between two connected computers in python?

description: Transfer a file between two connected computers in python


code:
#!/usr/bin/env python3
import sys
from configparser import ConfigParser
from io import StringIO

CONFIG = '''
[Box]
box.active = false
resolution_tracker.active = true
box.api_key =
box.api_secret =
box.job_interval = 480
box.max_attempts = 6
box.users =

[Google]
google.active = true
google.job_interval = 480
google.users = <useremail>
google.key_file_name = <key_file>
google.service_account_id = <account_id>
'''

parser = ConfigParser()
parser.readfp(StringIO(CONFIG))
parser.write(sys.stdout, space_around_delimiters=False)

rawDescription: Eliminating spaces between equal signs in ConfigParser - Python


code:
models/__init__.py

DBSession1 = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
DBSession2 = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
DBSession3 = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
DBSession4 = scoped_session(sessionmaker(extension=ZopeTransactionExtension()))
metadata1 = MetaData()
metadata2 = MetaData()
metadata3 = MetaData()
metadata4 = MetaData()

def initialize_sql(engines, drop_db=False):
    DBSession1.configure(bind=engine[0])
    DBSession2.configure(bind=engine[1])
    DBSession3.configure(bind=engine[2])
    DBSession4.configure(bind=engine[3])
    metadata1.bind = engine[0]
    metadata2.bind = engine[1]
    metadata3.bind = engine[2]
    metadata4.bind = engine[3]

rawDescription: Changing database per view & accessing multiple databases per view


rawDescription: counter modified during recursion in python

description: Counter modified during recursion in python


code:
output.write("My word sent is: {item}. "
             "The word divide in silabes is: {result}\n".format(item=item, result=result))

rawDescription: Sending and Receive data from a web page - Selenium WebDriver API


code:
target = [None] * 100
for _ in range(70):
    n = random.randint(0, 150)
    if n < 25:
        i = next(sub25)
    elif n < 100:
        i = next(from25to100)
    else:
        i = next(allOther)

    print(n, "goes to position", i)
    target[i] = n

rawDescription: Assigning the inputs, with respect to its len()


rawDescription: parsing objdump output

description: Parsing objdump output


code:
from django.db import connection
from django.contrib.contenttypes.models import ContentType
from tenant_schemas.middleware import TenantMiddleware
from tenant_schemas.utils import get_tenant_model, get_public_schema_name

def get_tenant_schemas():
    ''' Return all tenant schemas '''
    return get_tenant_model().objects.exclude(
        schema_name=get_public_schema_name())

class SchemaRouterMiddleware(TenantMiddleware):
    """
    Extends the original routing middleware from django-tenant-schemas.
    To support both types of deployment (cloud with many tenants vs. single
    tenant on own server) we check if our database holds more than one tenant:
     - Yes: Select schema based on subdomain (= django-tenant-schemas default)
     - No: Always use that single tenant's schema no matter what the
       incoming host name is (ip, domain, subdomain, ...)
    """
    single_tenant = False

    def __init__(self):
        '''
        Adding a new tenant will most likely go with a server restart so we can
        avoid querying the table with each request and only do this once here:
        '''
        tenants = get_tenant_schemas()
        if tenants.count() == 1:
            self.single_tenant = tenants[0]

    def process_request(self, request):
        if self.single_tenant:
            connection.set_tenant(self.single_tenant)
            request.tenant = self.single_tenant
            ContentType.objects.clear_cache()
        else:
            super(SchemaRouterMiddleware, self).process_request(request)

rawDescription: django-tenant-schemas: How to use with single tenant/without subdomains

description: Django-tenant-schemas: How to use with single tenant/without subdomains


code:
# the cache is serializing the result object behind the scenes
# it also works if we directly pickle and unpickle it
_d = dill.loads(dill.dumps(d))
_d
_d.A

rawDescription: Is there a way to pickle a scipy.interpolate.Rbf() object?


code:
filename = 'temps.txt'

all_temps = []

with open(filename, 'r') as f:
    for line in f:
        line = [s.strip() for s in line.split(',')]
        city = line[0]
        temps = [float(s) for s in line[1:] if s]
        all_temps.append((city, temps))

for city, temps in all_temps:
    print('city:{0}, temps:{1}'.format(city, temps))

rawDescription: Inputting data from file into tuple

description: Inputting data from file into tuple


code:
#!/usr/bin/env python3
from multiprocessing.dummy import Pool # use threads for I/O bound tasks
from urllib.request import urlretrieve

urls = [...]
result = Pool(4).map(urlretrieve, urls) # download 4 files at a time

rawDescription: How to parallelized file downloads?

description: Parallelized file downloads


rawDescription: generate random lognormal distributions using shape of observed data

description: Generate random lognormal distributions using shape of observed data


code:
groups = concat([df.groupby('A').head(3),df.groupby('A').tail(1)]).sort_index()
groups = groups[features]
groups.head(20)
groups.info()

rawDescription: Data processing with adding columns dynamically in Python Pandas Dataframe


code:
@decorator_that_makes_func_into_generator
def countdown(n):
    while n > 0:
        yield n
        n = n - 1

print(type(countdown(5)), list(countdown(5)))
# Outputs <class 'generator'> [5, 4, 3, 2, 1]

rawDescription: Using decorator to turn function into generator in python


code:
from collections import OrderedDict

d = OrderedDict()
import re

r1 = re.compile(r"(?<=\))\s")
r2 = re.compile(r"\(.*\)")
for line in lines.splitlines():
    key = tuple(r1.split(r2.findall(line)[0]))
      # (resid 44 and name H ) (resid 53 and name H ) ->  (resid 53 and name H ) (resid 44 and name H )
    rev_k = tuple(reversed(key))
    if rev_k in d:
        d[rev_k] += " DUPLICATE " + " ".join(line.rsplit(None,4)[1:])
    elif key in d:
        d[key] += " DUPLICATE " + " ".join(line.rsplit(None,4)[1:])
    else:
        d[key] = line

rawDescription: Python Find Duplicate with Note String Preservation


code:
import urllib2
f = urllib2.urlopen("http://api.hostip.info/get_html.php?ip=64.233.160.0")
data = f.read()
f.close()

rawDescription: How can I geolocate a bunch of IP addresses with Python?

description: Geolocate a bunch of IP addresses with Python


code:
import requests
requests.packages.urllib3.add_stderr_logger()
requests.get('https://www.google.com/')
requests.get('https://www.google.com/')

rawDescription: What is the practical difference between these two ways of making web connections in Python?


code:
import itertools as IT
import collections

list1 = ["INTJ","ENTJ","ESTJ"]
list2 = ["INTP","ESFJ","ISTJ"]

matches = collections.Counter()
for item1, item2 in zip(list1, list2):
    for n in range(1,4):
        for idx in IT.combinations(range(4), n):
            if all(item1[i] == item2[i] for i in idx):
                print(item1, item2, idx)
                matches[n] += 1

print(matches)

rawDescription: calculate combinations of letter-pair matches in two lists?

description: Calculate combinations of letter-pair matches in two lists


code:
def numbers(s):
    return int(re.search(r"\d+", s).group(0))

rawDescription: Filtering text in python for numeric values


code:
with d3py.NetworkXFigure(G, width=500, height=500) as p:
    p += d3py.ForceLayout()
    p.css['.node'] = {'fill': 'blue', 'stroke': 'magenta'}
    p.show()

rawDescription: How to plot graphs using networkx and d3py

description: Plot graphs using networkx and d3py


code: y = list(set(x))

rawDescription: Remove values from list leaving only unique values


rawDescription: wxPython auinotebook.GetSelection() return index to the first page

description: WxPython auinotebook.GetSelection() return index to the first page


code:
df = pd.DataFrame([[1, 2], [3, 4]], columns=['A', 'B']).set_index('A')
df
df.index.names = [None]
df

rawDescription: Pandas reindexing data frame issue

description: Pandas reindexing data frame issue


code: x = np.random.randint(0,3,size=1000)

rawDescription: Most efficient way to determine if a numpy masked array contains any non-masked values?


code: parent = relationship("Parent", backref=backref('children', lazy='noload'))

rawDescription: sqlalchemy add child in one-to-many relationship

description: Sqlalchemy add child in one-to-many relationship


code: {{ value|private:'_id' }}

rawDescription: Django Templates and MongoDB _id

description: Django Templates and MongoDB _id


code:
outputs
[' string ']

rawDescription: How to tree-search an html file tag with no specific type

description: Tree-search an html file tag with no specific type


code:
import re, base64
import os

folder = '/home/user/Public'
matcher = re.compile("<Message>(?P<text>[^<]*)<End Message>")

for filename in os.listdir(folder):
    infilename = os.path.join(folder, filename)
    if not os.path.isfile(infilename): continue

    base, extension = os.path.splitext(filename)
    filein = open(infilename, 'r')
    fileout = open(os.path.join(folder, '{}_edit.{}'.format(base, extension)), 'w')
    for line in filein:
        match = matcher.search(line)
        if match:
            fileout.write("<message> " + base64.b64encode(match.group('text').strip()) + " <End message>\n")
        else:
            fileout.write(line)

    filein.close()
    fileout.close()

rawDescription: Python - extract and modify part of a specific line of text with a function for all files in folder


code:
import re

def splitup(string):
    match = re.search(" \\d[^ ]*, ", string)
    if match is None:
        raise ValueError("Not a valid string: %r" % string)
    street = string[:match.start()]
    number = string[match.start(): match.end()].strip(", ")
    state = string[match.end():]
    return [street, number, state]

rawDescription: Python Splitting string by number and space


code:
import numpy as np
dists = np.vstack(([x_dists.T], [y_dists.T])).T

rawDescription: Joining two 2D numpy arrays into a single 2D array of 2-tuples


rawDescription: Scrapy crawling stackoverflow questions matching multiple tags


code:
database_set = dict()
for row in database: #Loop on the smaller file so we store less in memory.
    database_set[row[3]] = (row[0],row[1],row[2])
for line in data:
    if line[3] in database_set:
        row = database_set[line[3]]

        result.writerow((line[0],line[1],line[2],row[0],row[1],row[2],line[3]))

rawDescription: comparing two large csv file and writing another one with python

description: Comparing two large csv file and writing another one with python


code: form = MyModelForm (initial={'gender':'M'})

rawDescription: How do I make a Django ModelForm menu item selected by default?

description: Make a Django ModelForm menu item selected by default


code:
['Local Area Connection* 12',
 'Bluetooth Network Connection',
 'Wi-Fi',
 'Ethernet',
 'VirtualBox Host-Only Network',
 '(unknown)',
 'isatap.{4E4150B0-643B-42EA-AEEA-A14FBD6B1844}',
 'isatap.{BB05D283-4CBF-4514-B76C-7B7EBB2FC85B}']

rawDescription: How to get meaningful network interface names instead of GUIDs with netifaces under Windows?

description: Get meaningful network interface names instead of GUIDs with netifaces under Windows


code:
from functools import reduce

results = [reduce(lambda v, f: f(v), fnl, element) for element in content]

rawDescription: Python: apply list of functions to each element in list


code:
import socket
import sys
s = socket.socket()
s.bind(('', 8080))
s.listen(3)

i = 0

while True:

    #Accept connections from the outside
    (clientsocket, address) = s.accept()
    print(address)
    i = i + 1
    with open('file_' + str(i) + ".xml", 'wb') as f:
        while True:
            l = clientsocket.recv(1024)
            if not l:
                break
            f.write(l)

    clientsocket.close()

s.close()

rawDescription: Send/receive file over LAN with C# to/from Python


rawDescription: How to get count of unpublished commit with GitPython?

description: Get count of unpublished commit with GitPython


code:
L = ['Explain', 'it', 'to', 'me', 'please']
result = [L[i:j] for i in xrange(len(L)) for j in xrange(i + 1, len(L) + 1)]
result.sort(key=lambda x: (-len(x), x))
pprint.pprint(result)

rawDescription: Get all combinations of neighbour elements in list


rawDescription: Finding the maximum height of a tree and returning the sum of that path


code: d.apply(lambda row: min([row['A'], row['B']])-row['C'], axis=1)

rawDescription: Adding calculated column(s) to a dataframe in pandas


code:
import argparse

test_value = None

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('-t', '--test_value', dest='test_value', required=True, default="sample value")
    args = parser.parse_args()
    return args

def configure():
    global test_value
    args = parse_args()
    test_value = args.test_value

rawDescription: how can I avoid using a global variable as a parameter to a python decorator

description: Avoid using a global variable as a parameter to a python decorator


code:
line = line[:-2]
line = line.rstrip()

rawDescription: extracting data from multiple text files using python

description: Extracting data from multiple text files using python


code:
import time, thread, sys, threading

time_passed = 0
l = []

def input(l):
    """ Waits for a keypress """
    raw_input()
    l.append(None)

def timer():
    """ Prints out a number every millisecond """
    global time_passed
    time_passed += 1
    sys.stdout.write(format(time_passed)+"\r")
    sys.stdout.flush()

def running(state):
    """ Toggles play/pause state if Enter is pressed """
    global l
    while True:
        time.sleep(.1)
        if state:
            timer()
        if l:
            l = []
            state = not state
            return state

def main():
    """ Waits for a keypress while either running or stopping the timer """
    global l
    state = True
    while True:
        thread.start_new_thread(input, (l,))
        state = running(state)

main()

rawDescription: multithreading in Python: extremely high CPU consumption

description: Multithreading in Python: extremely high CPU consumption


code:
class Resource(object):

    def GET(self, request, **kwargs):
        return NotImplemented()

    def HEAD(self, request, **kwargs):
        return NotImplemented()

    def POST(self, request, **kwargs):
        return NotImplemented()

    def DELETE(self, request, **kwargs):
        return NotImplemented()

    def PUT(self, request, **kwargs):
        return NotImplemented()

    def __call__(self, request, **kwargs):
        handler = getattr(self, request.method)
        return handler(request, **kwargs)

rawDescription: MongoKit vs MongoEngine vs Flask-MongoAlchemy for Flask


rawDescription: Couple of points that are closer to each other in a list


code:
def access_with_js(ctx, route):
    if len(route) == 0:
        raise Exception("route must have at least one element")
    accessor_string = route[0]
    for elem in route[1:]:
        if type(elem) in [str, unicode]:
            accessor_string += "['" + elem + "']"
        elif type(elem) == int:
            accessor_string += "[" + str(elem) + "]"
        else:
            raise Exception("invalid element in route, must be text or number")
    return ctx.eval(accessor_string)

def get_py_obj(ctx, obj, route=[]):
    def dict_is_empty(dict):
        for key in dict:
            return False
        return True

    def access(obj, key):
        if key in obj:
            return obj[key]
        return None

    cloned = None
    if isinstance(obj, list) or isinstance(obj, PyV8.JSArray):
        cloned = []
        temp = str(access_with_js(ctx, route)) #working around a problem with PyV8 r429
        num_elements = len(obj)
        for index in range(num_elements):
            elem = obj[index]
            cloned.append(get_py_obj(ctx, elem, route + [index]))
    elif isinstance(obj, dict) or isinstance(obj, PyV8.JSObject):
        cloned = {}
        for key in obj.keys():
            cloned_val = None
            if type(key) == int:
                #workaround for a problem with PyV8 where it won't let me access
                #objects with integer accessors
                val = None
                try:
                    val = access(obj, str(key))
                except KeyError:
                    pass
                if val == None:
                    val = access(obj, key)
                cloned_val = get_py_obj(ctx, val, route + [key])
            else:
                cloned_val = get_py_obj(ctx, access(obj, key), route + [key])
            cloned[key] = cloned_val
    elif type(obj) == str:
        cloned = obj.decode('utf-8')
    else:
        cloned = obj
    return cloned

def get_js_obj(obj):
    #workaround for a problem with PyV8 where it will implicitely convert python lists to js objects
    #-> we need to explicitely do the conversion. see also the wrapper classes for JSContext above.
    if isinstance(obj, list):
        js_list = []
        for entry in obj:
            js_list.append(get_js_obj(entry))
        return PyV8.JSArray(js_list)
    elif isinstance(obj, dict):
        js_obj = {}
        for key in obj.keys():
            js_obj[key] = get_js_obj(obj[key])
        return js_obj
    else:
        return obj

rawDescription: Converting Python objects to JavaScript for PyV8


code:
# -*- coding: utf-8 -*-

TWOPOWER32 = 1 << 32
TWOPOWER64 = 1 << 64

TWOPOWER31 = TWOPOWER32 >> 1
TWOPOWER63 = TWOPOWER64 >> 1

def uint32(value):
    assert 0 <= value < TWOPOWER32
    return (
        value >> 0 & 255,
        value >> 8 & 255,
        value >> 16 & 255,
        value >> 24 & 255
    )

def uint64(value):
    assert 0 <= value < TWOPOWER64
    return (
        value >> 0 & 255,
        value >> 8 & 255,
        value >> 16 & 255,
        value >> 24 & 255,
        value >> 32 & 255,
        value >> 40 & 255,
        value >> 48 & 255,
        value >> 56 & 255
    )

def int32(value):
    assert -TWOPOWER31 <= value < TWOPOWER31
    return uint32((TWOPOWER32 + value) & (TWOPOWER32-1))

def int64(value):
    assert -TWOPOWER63 <= value < TWOPOWER63
    return uint64((TWOPOWER64 + value) & (TWOPOWER64-1))

__all__ = ['uint32', 'int32', 'uint64', 'int64']

rawDescription: position contents of array.array into heap

description: Position contents of array.array into heap


code:
def func(l, index):
   for i in range( index):
      if l[index - i - 1] > l[index]:
         break
   else:
       return 0 # return 0 if it never finds a larger number (loop completes)

   return i + 1 # this should be the distance

rawDescription: How to find index of previous data that is higher than current value with pandas?

description: Find index of previous data that is higher than current value with pandas


code:
out = list(sherwood(2))
for x in out: print(x)
out = list(sherwood(4))
for x in out: print(x)

rawDescription: Need a clever way to create a large list for a pygame/pyOpenGL project


code:
for num in range(999, -1, -1):
    print(i)

rawDescription: iterating over a reversed sequence of numbers

description: Iterating over a reversed sequence of numbers


code:
import sys
for item in r:
    text = item['text'] if 'text' in item else item
    sys.stdout.buffer.write(text.encode('utf-8'))
    sys.stdout.buffer.write('\n')

rawDescription: Need help Piping Python twitter script to NLP Bash script (Sed, grep etc....)


rawDescription: Determining whether a condition holds for all members of a list


code:
product = None
from itertools import *
if not product:
    def product(*args, **kwds):
        # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
        # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
        pools = map(tuple, args) * kwds.get('repeat', 1)
        result = [[]]
        for pool in pools:
            result = [x+[y] for x in result for y in pool]
        for prod in result:
            yield tuple(prod)

rawDescription: Suggestions for writing this small piece of code more elegantly


rawDescription: How to use a while loop to start at the right end of the list

description: Use a while loop to start at the right end of the list


code:
class userData(ndb.Model):
    id = ndb.StringProperty()
    name = ndb.StringProperty()
    emailAddress = ndb.StringProperty()

user = userData.query().filter(ndb.GenericProperty('id') ==  "requiredId")\
                       .fetch(projection=[userData.id, userData.name])

rawDescription: google app engine - ndb query to only get a few columns in python

description: Google app engine - ndb query to only get a few columns in python


code:
class FooBar(Foo):

    @property
    def bar(self):
        # return the same value
        # as in the base class
        return Foo.bar.fget(self)

rawDescription: How to call a property of the base class if this property is being overwritten in the derived class?

description: Call a property of the base class if this property is being overwritten in the derived class


code:
row = df.ix[0]  # also a copy
row['Day'] = 2

rawDescription: Why doesn't this function "take" after I iterrows over a pandas DataFrame?


code:
subpieces = pieces[0].split('_')
subpieces

rawDescription: strip text to create list and compare 2 similar lists

description: Strip text to create list and compare 2 similar lists


code:
df.groupby('categories').agg(f)
"""
              groups  end start
               first last first
categories
0            (5, 10]   17     1
1           (10, 15]   30    18
2            (5, 10]   42    31
"""

rawDescription: Pandas - group by consecutive ranges

description: Pandas - group by consecutive ranges


code:
for form in soup.find_all('form'):
    print(form.attrs.get('action'))

rawDescription: Trying to send a POST-request with urllib.request returns the same page


code:
def __ne__(self, other):
  return not self == other

rawDescription: __lt__ instead of __cmp__

description: __lt__ instead of __cmp__


code: XML('Line 1<br />Line 2')

rawDescription: multi line headers with web2py's SQLFORM.grid

description: Multi line headers with web2py's SQLFORM.grid


code:
frequncy_str = ('Lateral', 'Daily', 'Weekly', 'Monthly')
re_str = '(?i)<h5>(' + '|'.join(frequency_str) + ')</h5>'

rawDescription: Python macro in regular expression

description: Python macro in regular expression


code: new_dict = {k:v for (k, v) in self.request.jsondata.iteritems() if k in filter_props}

rawDescription: How do I filter a dict to match the attributes of my ndb.Model subclass

description: Filter a dict to match the attributes of my ndb.Model subclass


code:
s = "Hello(There|World!!"
re.split(r'[(|]+', s)

rawDescription: How can I use re's special characters when splitting a string?

description: Use re's special characters when splitting a string


code:
from core import functions

def simple_action():
    functions.core_function()

rawDescription: python module organisation

description: Python module organisation


code: a == Bird.default

rawDescription: Create current class default instance in its base class


code:
CC_agelist = [int(D['AGE']) for D in Company.itervalues() if D['Company Car'] == 'Yes']
mean_CC_age = float(sum(CC_agelist)) / len(CC_agelist)

rawDescription: Directly calculating conditional averages in a Python Dictionary


rawDescription: python - please help cant figure out how to add up user input of an array

description: Python - please help cant figure out how to add up user input of an array


code:
import myscript
# this script only creates the list, not modifying it at all

if __name__ == '__main__':
    myscript.create_list()

rawDescription: How to combine two Python scripts?

description: Combine two Python scripts


rawDescription: How to read a CSV by column name using pandas in python?

description: Read a CSV by column name using pandas in python


code:
def self_reference(f):
    f.func_defaults = f.func_defaults[:-1] + (f,)
    return f

@self_reference
def foo(verb, adverb='swiftly', self=None):
    return '%s %s %s' % (self.subject, verb, adverb)

rawDescription: how can python function access its own attributes?

description: How can python function access its own attributes


rawDescription: How to divide sentences based on some special contents, Python,

description: Divide sentences based on some special contents, Python,


code:
class Person(object):
    balance = Integer('_balance')
    age = PositiveInteger('_age')

rawDescription: Reusable validating class attributes

description: Reusable validating class attributes


code:
scope = {'bar': bar, 'foo':foo}
expression = 'bar(42)'
print('eval({!r}, scope) returns {}'.format(expression, eval(expression, scope)))

rawDescription: How do I execute a function inside a namespace dict?

description: Execute a function inside a namespace dict


code: np.column_stack([chl.data for chl in channels])

rawDescription: Fast way to transpose np.arrays and write them to file


code: shutil.copyfile(item, 'INPUT')

rawDescription: Iterate through a list of files and copy them using python


code:
data = [
    ['Basket', 'NBA ET', 'Kobe'],
    ['Basket', 'NCAA', 'Shaq'],
    ['Basket', 'ENG', 'Shaq'],
]
drop = ['NBA', 'NCAA']
[d for d in data if not any(dropped in d[1].split() for dropped in drop)]
drop = set(drop)
[d for d in data if not drop.intersection(d[1].split())]

rawDescription: Filtering a list based on another list

description: Filtering a list based on another list


code:
import collections

class TransformedDict(collections.MutableMapping):
    """A dictionary that applies an arbitrary key-altering
       function before accessing the keys"""

    def __init__(self, *args, **kwargs):
        self.store = dict()
        self.update(dict(*args, **kwargs))  # use the free update to set keys

    def __getitem__(self, key):
        return self.store[self.__keytransform__(key)]

    def __setitem__(self, key, value):
        self.store[self.__keytransform__(key)] = value

    def __delitem__(self, key):
        del self.store[self.__keytransform__(key)]

    def __iter__(self):
        return iter(self.store)

    def __len__(self):
        return len(self.store)

    def __keytransform__(self, key):
        return key

rawDescription: Python: How to "perfectly" override a dict


rawDescription: How to change text in specific column in panda data frame?

description: Change text in specific column in panda data frame


code: {{ obj|get_class }}

rawDescription: How to get class in django template

description: Get class in django template


code:
def foo(x):
    print("foo: " + x)

def bar(x):
    print("bar: " + x)

f = foo
f("one")
f = bar
f("ten")

rawDescription: substitute of function pointers in python

description: Substitute of function pointers in python


rawDescription: Store additional keys in dict of dicts

description: Store additional keys in dict of dicts


code:
import re

# \b is the word boundary assertion, so it requires that there be a word
# followed by non-word character (or vice-versa) at that point
# This assumes none of your search strings begin or end with non-word characters
pats1 = [re.compile(r'\b{}\b'.format(re.escape(x))) for x in list1]

for item, pat in zip(list1, pats1):
    if pat.search(sring1):
        print(item)

rawDescription: Searching in a string in python

description: Searching in a string in python


code:
for i, (a, b) in enumerate(tuple_list):
    new_b = some_process(b)
    tuple_list[i] = (a, new_b)

rawDescription: Iterate a list of tuples

description: Iterate a list of tuples


rawDescription: Where to place the code to remove a tkinter widget from a window


code:
searchKey = [2,4,10,5,8,9]  # Add more elements for testing
setSearchKey = set(searchKey)
ismember = [setSearchKey.issuperset(row) for row in x.tolist()]

rawDescription: check if numpy array is subset of another array

description: Check if numpy array is subset of another array


code: unicodedata.lookup("WHITE SCISSORS")

rawDescription: Inverse of Python's unicodedata.name?

description: Inverse of Python's unicodedata.name


code:
import a
print(a.lis[0]) # will print first value in list in a.py
for elem in a.lis:
    print(elem ** 2) # will print square of every value in list in a.py

rawDescription: How to get data from a.py to import into b.py

description: Get data from a.py to import into b.py


code:
def key_fix_decorator(f):
    @wraps(f, assigned=('__name__', '__doc__'))
    def wrapped(self, *args, **kwargs):
        if args and isinstance(args[0], str):
            args = (args[0].lower(),) + args[1:]
        return f(self, *args, **kwargs)
    return wrapped

rawDescription: Why doesn't my idea work in python2?

description: Why doesn't my idea work in python2


code: frame = frame[frame.index >= frame[(~frame.A.isnull()) & (~frame.B.isnull()) & (~frame.C.isnull())].head(1).index]

rawDescription: pandas dataframe: index is DatetimeIndex and 4 columns. Some NaN

description: Pandas dataframe: index is DatetimeIndex and 4 columns. Some NaN


code:
# x = input data, size(<points>, <dimensions>)

# fit the full model
max_components = x.shape[1] # as many components as input dimensions
pca = PCA(n_components=max_components)
pca.fit(x)

# transform the data (contains all components)
y_all = pca.transform(x)

# keep only the top k components (with greatest variance)
k = 2
y = y_all[:, 0:k]

rawDescription: How to change parameters of a scikit learn function dynamically i.e. find best parameter

description: Change parameters of a scikit learn function dynamically i.e. find best parameter


rawDescription: Django - Convert value get from HTML select in POST request back to number before validate ModelForm


code:
dict1 = {data[u'dispURL']:int(data[u'number'])   for data in dict1[u'd'][u'results']}
dict2 = {data[u'url']:int(data[u'pos'])          for data in dict2[u'result_page']}
dict3 = {data[u'displayurl']:int(data[u'index']) for data in dict3[u'hits']}

rawDescription: Comparing values of different python dictionaries with different keys & structures


code:
convertToNumber('foo bar baz')
x = _
convertFromNumber(x)

rawDescription: String of text to unique integer method?

description: String of text to unique integer method


code:
class Foo:
    def __init__(self):
        self.x = 10
    def __getattr__(self, name):
        return name

f = Foo()
f.x    # -> 10
f.bar   # -> 'bar'

rawDescription: What are the differences amongst Python's "__get*__" and "_del*__" methods?


code:
step=0.1
x= np.arange(-5,5, step=step)

f1 = x**2  - 4
f2 = -x**2 + 4

dif= f2-f1
dif_sign= dif/np.abs(dif) #just the sign of the function
dif_sign_dif= np.diff(dif_sign) #places where the sign changes
change_idxs= np.where(dif_sign_dif!=0)[0] #indexes where the sign changes
print -5+change_idxs*step

rawDescription: Intersection of 2 plots (sage)

description: Intersection of 2 plots (sage)


code:
test = Image.open('test.png')
test.close()

rawDescription: How to close Pillow opened image - Python

description: Close Pillow opened image - Python


code: names = Person.objects.values_list('name', flat=True)

rawDescription: Convert One Key Value from List of Dictionaries to List


code:
def bytes_list_test_opt():
    tStart = time.clock()
    bs = b''.join(itertools.repeat(MSG, COUNT))
    print('byte list opt time:', time.clock() - tStart)

rawDescription: Python bytearray verses list of bytes

description: Python bytearray verses list of bytes


rawDescription: Pandas Sort Order of Columns by Row

description: Pandas Sort Order of Columns by Row


code:
q = Session.query(Articles).filter(Articles.article_id == ArticleTags.article_id).\
    filter(ArticleTags.tag_id == Tags.tag_id).\
    filter(Tags.name == 'tag_name')

rawDescription: How to do a JOIN in SQLAlchemy on 3 tables, where one of them is mapping between other two?

description: Do a JOIN in SQLAlchemy on 3 tables, where one of them is mapping between other two


code:
# Size of regular_sequence
N = regular_sequence.size

# Use cumsum to pre-compute start of every occurance of regular_sequence
offset_arr = np.cumsum(lag_seq)
idx = np.arange(offset_arr.size)*N + offset_arr

# Setup output array
out = np.zeros(idx.max() + N,dtype=regular_sequence.dtype)

# Broadcast the start indices to include entire length of regular_sequence
# to get all positions where regular_sequence elements are to be set
np.put(out,idx[:,None] + np.arange(N),regular_sequence)

rawDescription: Generate 1d numpy with chunks of random length


code: ['green']

rawDescription: Search between two files in Python

description: Search between two files in Python


code:
import theano, numpy
T = theano.tensor.matrix()
f = theano.function([T], T[:, [1, 2, 4]])
f(numpy.arange(25).reshape(5, 5))

rawDescription: Theano Tensor remove element?

description: Theano Tensor remove element


code:
import pandas as pd
import numpy as np

pd.options.display.mpl_style = 'default'

# generate some artificial data
# ======================================
date_rng = np.random.choice(pd.date_range('2012-01-01', '2015-12-31', freq='D'), 2000)
val = np.random.randint(1, 10, size=2000)
df = pd.DataFrame({'Date': date_rng, 'Credit':val}).set_index('Date').sort_index()

# before: same problem as you saw
# ======================================
df.resample('Q', how=sum).plot(kind='bar')

rawDescription: easily make labels look nice pandas plotting

description: Easily make labels look nice pandas plotting


code:
import logging
root = logging.getLogger()
root.setLevel(logging.INFO)
root.addHandler(logging.StreamHandler())
logging.error("test")
logging.debug("test")

rawDescription: Should python debug statements etc be wrapped by if debug?


code:
f.x = 17
b = f
del f
b()

rawDescription: how can python function access its own attributes?


code:
#Django model utils TimeStampedModel
class TimeStampedModel(models.Model):
    """
    An abstract base class model that provides self-updating
    ``created`` and ``modified`` fields.

    """
    created = AutoCreatedField(_('created'))
    modified = AutoLastModifiedField(_('modified'))

    class Meta:
        abstract = True

class Location(TimeStampedModel):
    """
    Add additional fields
    """

rawDescription: updated at field in django model

description: Updated at field in django model


code: train, test = data.loc[mask], data.loc[~mask]

rawDescription: Date Time Selection Python

description: Date Time Selection Python


code:
[tbl._cells[0, col]._text.set_color('white') for col in range(len(colors.columns))]
plt.show()

rawDescription: Conditionally change background color of specific cells


code:
(u'2011-12-19 08:38', u'R. Fernandes Vieira; esquina Prot\xe1sio Alves', u'-30.035535,-51.211079', u'\xfanico', u'automotores', u'sem\xe1foro 30-70', u'3')
(u'2011-12-17 16:00', u'Av. Prot\xe1sio Alves; esquina Ramiro Barcelos', u'-30.036916,-51.208093', u'bairro-centro', u'automotores', u'semaforo 50-15', u'2+c')

rawDescription: Convert JSON to SQLite in Python - How to map json keys to database columns properly?


rawDescription: Event handling with Jython & Swing

description: Event handling with Jython & Swing


rawDescription: Django: Map data from an external APIs into a model?


code:
import hashlib

for attr in dir(hashlib):
    if attr not in ('__name__', '__file__'):
        delattr(hashlib, attr)

hashlib = imp.reload(hashlib)

rawDescription: Why does python's imp.reload() not delete old classes and functions?


code:
class Image(models.Model):
    name = models.CharField(max_length=50)

class ImageRevision(models.Model):
    parent = models.ForeignKey(Image, related_name='revisions')
    image = models.ImageField(upload_to='images')
    created = models.DateTimeField(auto_now_add=True)

    class Meta:
        ordering = ['-created']
        get_latest_by = 'created'

rawDescription: Parent-child relationship efficiency

description: Parent-child relationship efficiency


code: self._matches(found.string, self.text)

rawDescription: Beautifulsoup find element by text using `find_all` no matter if there are elements in it


code: array([False,  True,  True,  True, False], dtype=bool)

rawDescription: Opencv: How to detect pixel color change


code:
def change_filename(name):
    filename = name.split('.')
    wordlist = filename[:-4]
    extension = '.' + filename[-1]
    wordlist[-1] = '{' + wordlist[-1] + '}'
    os.rename(name, ' '.join(wordlist) + extension)

rawDescription: Rename files with Python

description: Rename files with Python


code:
def setup_worker():
    global clockExp, a

    clockStart = dt.datetime.now()
    clockExp = clockStart + dt.timedelta(seconds=900)
    a = getRenewed()

def worker(item):
    global clockExp, a

    clockCur = dt.datetime.now()
    clockRem = (clockExp - clockCur).total_seconds()

    if clockRem < 5: # renew with 5 seconds left
        clockStart = dt.datetime.now()
        clockExp = clockStart + dt.timedelta(seconds=900)
        a = getRenewed()

    f(item, a)

def main(L):
    pool = multiprocessing.Pool(initializer=setup_worker)

    pool.map(worker, L)

rawDescription: Multiprocessing subject to timer

description: Multiprocessing subject to timer


code: x, y = map(os.getpid, ("process1", "process2"))

rawDescription: Dynamic method in python

description: Dynamic method in python


code:
Characters = [Percy,Annabeth,Leo,Chuck,Sarah]
random.shuffle(Characters)

rawDescription: Select unique value from list

description: Select unique value from list


code:
personage = Personage.objects.all()[0]
personage.first_name

rawDescription: approaching django model fields through a field name mapping

description: Approaching django model fields through a field name mapping


code:
data = {'numeric_1':[12.1, 3.2, 5.5, 6.8, 9.9],
        'categorical_1':['A', 'B', 'C', 'B', 'B']}
frame = pd.DataFrame(data)
dummy_values = pd.get_dummies(data['categorical_1'] + ['D','E'])

rawDescription: Specify list of possible values for Pandas get_dummies


rawDescription: Joining a list of dicts

description: Joining a list of dicts


code:
df["Minimum_Amount"] = df.groupby("Name")["Amount"].transform('min')
df

rawDescription: Finding multiple minimums and adding new column in pandas


code: m = min(seq, default=-1)

rawDescription: How to tell when more than one index matches?

description: Tell when more than one index matches


code:
import re

def findSubString(raw_string, start_marker, end_marker):
    return re.sub(
        r'(?<={}).*?(?={})'.format(re.escape(start_marker), re.escape(end_marker)),
        lambda m: m.group().strip().replace(' ', '_'),
        raw_string)

line1 = "Who acted as `` Bruce Wayne '' in the movie `` Batman Forever '' ?"
line1 = findSubString(line1, "``", "''")
assert line1 == "Who acted as ``Bruce_Wayne'' in the movie ``Batman_Forever'' ?"

rawDescription: Python find all matching substring patterns and replace substring


code:
result 1
result 2

rawDescription: Access data from autocomplete search form


code:
U = IndexedBase("U")
l = symbols("lambda")
var("mu u v w x y z i j k")

rawDescription: Expand index notation equation using sympy


code:
data
numpy.where( data > 3 )

rawDescription: Count all values in a matrix greater than a value


rawDescription: Naming Pandas Series while stacking from DataFrame


code: [(sum(lis[:i+1]),sum(lis[i-(w-1):i+1]))[i>(w-1)]  for i in range(len(lis))]

rawDescription: Python Running cumulative sum with a given window


code:
myheader = "3, 2, 3, 14, 4"
a = bitstring.pack(myheader, 1, 0, 5, 1000, 2)
a.bin
a.tobytes()

rawDescription: Accessing bitfields while reading/writing binary data structures


code:
class A():
    def __repr__(self):
        return "This is A"

rawDescription: How to invoke __repr__ of parent class?

description: Invoke __repr__ of parent class


code:
#!/usr/bin/env python

import sys
import json
import cgi

fs = cgi.FieldStorage()

sys.stdout.write("Content-Type: application/json")

sys.stdout.write("\n")
sys.stdout.write("\n")

result = {}
result['success'] = True
result['message'] = "The command Completed Successfully"
result['keys'] = ",".join(fs.keys())

d = {}
for k in fs.keys():
    d[k] = fs.getvalue(k)

result['data'] = d

sys.stdout.write(json.dumps(result,indent=1))
sys.stdout.write("\n")

sys.stdout.close()

rawDescription: AJAX Posting to Python cgi

description: AJAX Posting to Python cgi


code:
import string

def validate(alpha):
    # Is it exactly 26 characters long?
    if len(alpha) != 26: return False
    for c in alpha:
        # Is every character in [A-Z]?
        if c not in string.ascii_uppercase: return False
        # Is this character duplicated?
        if alpha.count(c) > 1: return False
    return True

alphabet = ""
while not validate(alphabet):
    alphabet = input("Please input the encryption alphabet in order (only A-Z"
                     " allowed, with no duplicates): ")

msg = input("Now input your encrypted message: ")

print("Your encrypted alphabet is:", alphabet)
print("Your encrypted message is:", msg)

# Create a mapping from one alphabet to the other using a dictionary
table = dict(zip(alphabet, string.ascii_uppercase))
cleartext = "".join([table[c] if c in table else c for c in msg])

print("Your decrypted message reads:", cleartext)

rawDescription: Decrypting Input?

description: Decrypting Input


code:
issubclass(type(o), str)
type(o) in ([str] + str.__subclasses__())

rawDescription: What's the canonical way to check for type in python?


code:
import subprocess
import sys
inputfile = open('input.txt', 'r')
inputfile.readline()
inputfile.readline()
p = subprocess.call(['python', 'program2.py'],stdin=subprocess.PIPE,stdout=sys.stdout)
p.communicate(inputfile.read())
inputfile.close()

rawDescription: How do I create a python subprocess using a file for stdin starting at a certain line?

description: Create a python subprocess using a file for stdin starting at a certain line


code:
import re

repeat = re.compile(r'(?P<start>[a-z])(?P=start)+-?')

rawDescription: Python split string by pattern

description: Python split string by pattern


code: getargspec(func)

rawDescription: Python function vs partial - Scipy doesn't like partials


code:
a_file = "path/to/a.csv"
b_file = "path/to/b.csv"

# read from file a.csv
with open(a_file) as f:
    # skip headers
    f.next()
    # get first colum as keys
    keys = (line.split(',')[0] for line in f)

# create empty dictionary:
d = {}

# read from file b.csv
with open(b_file) as f:
    # gather headers except first key header
    headers = f.next().split(',')[1:]
    # iterate lines
    for line in f:
        # gather the colums
        cols = line.strip().split(',')
        # check to make sure this key should be mapped.
        if cols[0] not in keys:
            continue
        # add key to dict
        d[cols[0]] = dict(
            # inner keys are the header names, values are columns
            (headers[idx], v) for idx, v in enumerate(cols[1:]))

rawDescription: How do you create nested dict in Python?

description: Create nested dict in Python


code:
rig = re.compile('[/.]')
rig.split(line)[-7:-1]

rawDescription: Python parse filenames by custom field pattern


code:
#!/usr/bin/python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

rawDescription: Set Python27 Google AppEngine default encoding for entire app


code:
from osgeo import gdal
ds = gdal.Open('path/to/file')
width = ds.RasterXSize
height = ds.RasterYSize
gt = ds.GetGeoTransform()
minx = gt[0]
miny = gt[3] + width*gt[4] + height*gt[5]
maxx = gt[0] + width*gt[1] + height*gt[2]
maxy = gt[3]

rawDescription: Obtain Latitude and Longitude from a GeoTIFF File


code:
win = Window().partitionBy('x').orderBy(F.col('count').desc())
result = (counts
          .withColumn('row_num', F.rowNumber().over(win))
          .where(F.col('row_num') == 1)
          .select('x', 'y')
         )
result.show()
## +---+---+
## |  x|  y|
## +---+---+
## |  0|  1|
## |  1|  1|
## +---+---+

rawDescription: Mode of grouped data in (py)Spark

description: Mode of grouped data in (py)Spark


code:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm

# fill grid
x = np.linspace(1,10,10)
y = np.linspace(1,10,10)

X, Y = np.meshgrid(x,y)
Z = np.abs(np.cos(X**2 - Y**2) * X**2 * Y)

# plot
f, ax = plt.subplots()
p = plt.pcolormesh(X, Y, Z, norm=LogNorm(), vmin=1e-2, vmax=1e2)
cb = plt.colorbar(p, ax=ax, orientation='horizontal', aspect=10)

# We need to nomalize the tick locations so that they're in the range from 0-1...
minorticks = p.norm(np.arange(1, 10, 2))
cb.ax.xaxis.set_ticks(minorticks, minor=True)

plt.show()

rawDescription: Minor ticks in matplotlib's colorbar

description: Minor ticks in matplotlib's colorbar


code:
@decorate_all_methods(Http500Resistant)
class A(object):
    ...

rawDescription: Try/Except Every Method in Class?

description: Try/Except Every Method in Class


code:
config = {
  'webapp2_extras.jinja2': {
    'template_path': 'templates',
    'filters': {
      'timesince': filters.timesince,
      'datetimeformat': filters.datetimeformat
    },
    'environment_args': {'extensions': ['jinja2.ext.i18n']},
  },
  'webapp2_extras.sessions': {'secret_key': 'my-secret-key'}
}

rawDescription: Organizing my config variable for webapp2


code:
import sys
sys.version

rawDescription: Is it possible to run a pypy kernel in the Jupyter notebook?


code:
import pandas as pd
df = pd.read_csv('D:\\Apps\\Coursera\\Kaggle-Titanic\\Data\\train.csv', header = 0)

rawDescription: Write to_csv with Python Pandas: Choose which column index to insert new data


code: import some_module

rawDescription: How importing works. Why imported modules not inheriting other imported modules


rawDescription: Combine files based on columns

description: Combine files based on columns


code:
for line in tNew.split('\n'):
    mail.write(line + '\n')

rawDescription: Add '\n' to each line of an open file in Python


code: array(randint(0, high=1<<32, size=10**8), uint32)

rawDescription: How to sort an array of integers faster than quicksort?

description: Sort an array of integers faster than quicksort


code:
def _compute(self, windows, dates, assets):
    # `windows` here is list of iterators yielding 2D slices of
    # the user's requested inputs

    # `dates` and `assets` are row/column labels for the final output.

    # Allocate a (dates x assets) output array.
    # Each invocation of the user's `compute` function
    # corresponds to one row of output.
    output = allocate_output()

    for i in range(len(dates)):

        # Grab the next set of input arrays.
        inputs = [next(w) for w in windows]

        # Call the user's compute, which is responsible for writing
        # values into `out`.
        self.compute(
            dates[i],
            assets,
            # This index is a non-copying operation.
            # It creates a view into row `i` of `output`.
            output[i],
            *inputs  # Unpack all the inputs.
        )

    return output

rawDescription: Quantopian / Zipline: weird pattern in Pipeline package


rawDescription: How to mock chained function calls in python?

description: Mock chained function calls in python


code:
import time
time

rawDescription: Where is time.py located?

description: Where is time.py located


code: df[df.a < df.a.quantile(.95)]

rawDescription: Eliminating all data over a given percentile


code: np.all(res1 == res2)

rawDescription: Slice numpy array based on x number of other arrays


code:
self.request.user
self.request.user.is_authenticated() # return True if the user is logged in

rawDescription: Django templates - Conditionally displaying a button for logged in users


code:
import sys
from filenameforthetrendclass import Trend

class Game(object):

    def __init__(self):
        self.twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)

    def do_something(self, user_name):
        a_trend = Trend(user_name)      # was originally 'game = '

def main():
    game = Game()
    game.do_something(sys.argv[1])

rawDescription: importing a class from another python file issue

description: Importing a class from another python file issue


code:
dest = bytearray(10) # all zero bytes
v = memoryview(dest)
struct.pack_into("2c", v[3:5], 0, b'\x07', b'\x08')
print(repr(dest))

rawDescription: Read into a bytearray at an offset?

description: Read into a bytearray at an offset


code:
def fun1(request):
    user_input = request.GET.get('value')
    #put your code here
    return HttpResponse('what you want to output to web')

rawDescription: Django ajax no refresh:django view without redirecting or refreshing a page


code:
first = queue.enqueue(first_job)
second = queue.enqueue(second_job, depends_on=first)

rawDescription: Passing results to depending on job - python rq


code:
from operator import itemgetter
lines.sort(key=itemgetter(2))

rawDescription: Python : Sort file by arbitrary column, where column contains time values


code:
subs = product("68", repeat=word.count("B"))
[reduce(lambda s, c: s.replace('B', c, 1), sub, word) for sub in subs]

rawDescription: Create strings with all possible combinations


code: type(bread1) == type(bread2)

rawDescription: One class with flags or many number of different classes


code:
for col in row[6:20]:
    foo()

rawDescription: Python: How to iterate over specific columns in rows in text file


code:
def get_pairs(lst, maxdiff):
    sl = sorted(lst) # may want to do lst.sort() if you don't mind changing lst
    count = 0
    i = 1
    N = len(sl)
    while i < N:
        # no need for abs -- we know the previous value is not bigger.
        if sl[i] - sl[i-1] <= maxdiff:
            count += 1
            i += 2 # these two values are now used
        else:
            i += 1
    return count

rawDescription: The fastest way to find all pairs of numbers in a list that differ not more than x


code:
import numpy as np
from scipy.optimize import leastsq

def cost_cpl(x, A, b):
    return (np.dot(A, x.view(np.complex128)) - b).view(np.double)

A = np.array([[1,1],[2j,0]],dtype=np.complex128)
b = np.array([1+3j,-2+2j],dtype=np.complex128)
init = np.array([0.0, 0.0, 0.0, 0.0])
x, r = leastsq(cost_cpl, init, args=(A, b))
print(x.view(np.complex128))

rawDescription: Scipy's leastsq with complex numbers

description: Scipy's leastsq with complex numbers


code:
for start in range(5, len(lst), 10):
    lst[start:start+5] = reversed(lst[start:start+5])

rawDescription: Python3 Make a list that increments for a certain amount, decrements for a certain amount


code: [x + num for num in y]

rawDescription: map lambda x,y with a constant x

description: Map lambda x,y with a constant x


rawDescription: How to only keep nodes in networkx-graph with 2+ outgoing edges or 0 outgoing edges?

description: Only keep nodes in networkx-graph with 2+ outgoing edges or 0 outgoing edges


code:
def foo():
   for x in range(5):
       yield x

f = foo()
f.__next__

rawDescription: how to check if an iterable allows more than one pass?

description: Check if an iterable allows more than one pass


code:
def find_kmers( arguments=sys.argv[1:] ):
    required_opts = ['-f','-c','-k']

    opts, args = getopt.getopt(arguments,'f:k:c:')

    opt_dic = dict(opts)

    for opt in required_opts:
        if opt not in opt_dic:
            return "incorrect arguments, please format as: python_script.py -f <filename> -k <kmer> -c <chromosome_name>"

    def rev_comp(sequence):
        reversed_dic = {'A':'T','T':'A','C':'G','G':'C'}
        return ''.join(reversed_dic[_] for _ in sequence[::-1])

    kmer = opt_dic['-k']
    # Replaces grep with temp file with trivial Python equivalent
    with open('S288C_R64.fasta') as f:
        chromosomes = [line[1:].strip() for line in f if '>' in line]

    # No need for any loop when just checking for exact value
    if opt_dic['-c'] not in chromosomes:
        return 'chromosome not found in %s. \n chromosomes in file are:%s'%(opt_dic['-f'],', '.join(str(_) for _ in chromosomes))

    def get_sequence(file):
        sequence = ''
        for line in file:
            if line.startswith('>'): break
            sequence += line.strip()
        return sequence.upper()

    with open(opt_dic['-f']) as ofile:
        for line in ofile:
            if line.startswith('>'):
                if line[1:].strip() == opt_dic['-c']:
                    sequence = get_sequence(ofile)
                    break

    kmer_matches1 = re.finditer('(?=%s)'%opt_dic['-k'],sequence)
    kmer_matches2 = re.finditer('(?=%s)'%opt_dic['-k'],rev_comp(sequence))

    def print_statement(start,strand):
        return '%s\thw1_script\tkmer=%s\t%s\t%s\t.\t%s\t.\tID=S288C;Name=S288C\n'%(opt_dic['-c'],opt_dic['-k'],start,start+len(opt_dic['-k'])-1,strand)

    pos_strand = collections.deque()
    neg_strand = collections.deque()
    for match1,match2 in itertools.izip(kmer_matches1,kmer_matches2):
        pos_strand.append(match1.start()+1)
        neg_strand.append(match2.start()+1)

    with open('answer.gff3','w') as wfile:
        while pos_strand and neg_strand:
            if pos_strand[0]<neg_strand[0]:
                start = pos_strand.popleft()
                wfile.write(print_statement(start,'+'))
            else:
                start = neg_strand.popleft()
                wfile.write(print_statement(start,'-'))

        for start in pos_strand:
            wfile.write(print_statement(start,'+'))
        for start in neg_strand:
            wfile.write(print_statement(start,'-'))

    return 'percent-GC = %s'%str(sum(sequence.count(gc) for gc in ["G","C"])/float(len(sequence)))

rawDescription: strange output when using flags in python

description: Strange output when using flags in python


code:
import nltk
nltk.__version__
pattern = r'''(?x)               # set flag to allow verbose regexps
              ([A-Z]\.)+         # abbreviations, e.g. U.S.A.
              | \$?\d+(\.\d+)?%? # numbers, incl. currency and percentages
              | \w+([-']\w+)*    # words w/ optional internal hyphens/apostrophe
              | [+/\-@&*]        # special characters with meanings
            '''

from nltk.tokenize.regexp import RegexpTokenizer
tokeniser=RegexpTokenizer(pattern)
line="My weight is about 68 kg, +/- 10 grams."
tokeniser.tokenize(line)

rawDescription: Python: Regular Expression not working properly


code:
import re
test = '"Properties" : { "Col_Row": "1 - 145", ... "Col_Row": "130 - 240" ... }}'
def repl(m):
  num = "%d%d" % (int(m.group(1))+1000,int(m.group(2))+1000)
  string = '"Geo_ID": "%s", "Col_Row": "%s - %s",' % (num,m.group(1),m.group(2))
  return string
output = re.sub(r'"Col_Row": "(.*?) - (.*?)",', repl, test)

rawDescription: How can I search & replace numbers with numbers+1000

description: Search & replace numbers with numbers+1000


rawDescription: Can anybody explain me the numpy.indices()?


rawDescription: Negative pipe matching with regular expressions


code:
v = np.array([1, 1, 1])
u_list = [v]

print(id(v))
for i in range(2):
    v = v + np.array([i, i, i])
    u_list.append(v)
    print(id(v))

rawDescription: Python Numpy Array Operator x += y Different From x = x + y?


code:
class ValuesWithCommentsConfigParser(ConfigParser.ConfigParser):

    def _read(self, fp, fpname):
        from ConfigParser import DEFAULTSECT, MissingSectionHeaderError, ParsingError

        cursect = None                        # None, or a dictionary
        optname = None
        lineno = 0
        e = None                              # None, or an exception
        while True:
            line = fp.readline()
            if not line:
                break
            lineno = lineno + 1
            # comment or blank line?
            if line.strip() == '' or line[0] in '#;':
                continue
            if line.split(None, 1)[0].lower() == 'rem' and line[0] in "rR":
                # no leading whitespace
                continue
                # continuation line?
            if line[0].isspace() and cursect is not None and optname:
                value = line.strip()
                if value:
                    cursect[optname].append(value)
            # a section header or option header?
            else:
                # is it a section header?
                mo = self.SECTCRE.match(line)
                if mo:
                    sectname = mo.group('header')
                    if sectname in self._sections:
                        cursect = self._sections[sectname]
                    elif sectname == DEFAULTSECT:
                        cursect = self._defaults
                    else:
                        cursect = self._dict()
                        cursect['__name__'] = sectname
                        self._sections[sectname] = cursect
                        # So sections can't start with a continuation line
                    optname = None
                # no section header in the file?
                elif cursect is None:
                    raise MissingSectionHeaderError(fpname, lineno, line)
                # an option line?
                else:
                    mo = self._optcre.match(line)
                    if mo:
                        optname, vi, optval = mo.group('option', 'vi', 'value')
                        optname = self.optionxform(optname.rstrip())
                        # This check is fine because the OPTCRE cannot
                        # match if it would set optval to None
                        if optval is not None:
                            optval = optval.strip()
                            # allow empty values
                            if optval == '""':
                                optval = ''
                            cursect[optname] = [optval]
                        else:
                            # valueless option handling
                            cursect[optname] = optval
                    else:
                        # a non-fatal parsing error occurred.  set up the
                        # exception but keep going. the exception will be
                        # raised at the end of the file and will contain a
                        # list of all bogus lines
                        if not e:
                            e = ParsingError(fpname)
                        e.append(lineno, repr(line))
            # if any parsing errors occurred, raise an exception
        if e:
            raise e

        # join the multi-line values collected while reading
        all_sections = [self._defaults]
        all_sections.extend(self._sections.values())
        for options in all_sections:
            for name, val in options.items():
                if isinstance(val, list):
                    options[name] = '\n'.join(val)

rawDescription: Python Config parser read comment along with value


code:
from collections import Counter
myDict = {'abc':'AGCTAC', 'def': 'AGGTAC', 'ghi':'AGGTAG'}
result = [Counter(site) for site in zip(*myDict.values())]
result
result[0]  # genes at site 0
result[2]  # genes at site 2

rawDescription: Iterating through tuples to count values in list


code:
tmp = (lst[i:i+5]+['\n'] for i in xrange(0, len(lst), 5))
[item for slist in tmp for item in slist][:-1]

rawDescription: What's the pythonic way to add a certain character after every X elements of a list?


code:
pattern = r'''
    (?P<name>.+?)\n
    SKU\s\#\s+(?P<sku_hidden>\d+)\n
    Quantity:\s+(?P<quantity>\d+)\n
    Gift\sWrap:\s+(?P<gift_wrap>.+?)\n
    Shipping\sMethod:.+?\n
    Price:.+?\n
    Total:\s+(?P<total_price>\$[\d.]+)
    '''

rawDescription: Regex not grabbing all groups, not working in multiline


code: results = [mutate(original_string) for _ in range(100)]

rawDescription: Random DNA mutation Generator

description: Random DNA mutation Generator


code:
import scrapy
from scrapy_splash import SplashRequest

class CheckSpider(scrapy.Spider):
    name = "scrapy-splash-example"
    handle_httpstatus_list = [404,429,503]

    start_urls = [
        "http://rads.stackoverflow.com/amzn/click/B00PRH5UJW",
        "http://rads.stackoverflow.com/amzn/click/B00KFITEV8",
        "http://rads.stackoverflow.com/amzn/click/B00J0T73XO",
        "http://rads.stackoverflow.com/amzn/click/B00O65Z0RS",
        "http://rads.stackoverflow.com/amzn/click/B00N3DDDRI"
    ]

    def start_requests(self):
        for url in self.start_urls:
            yield SplashRequest(url,
                                callback=self.parse,
                                args={
                                    'wait': 1,
                                })

    def parse(self, response):
        self.logger.debug("Response: status=%d; url=%s" % (response.status, response.url))

rawDescription: Scrapy bypassing start_urls

description: Scrapy bypassing start_urls


code:
def get_parallel_paths(G):
    return [list(nx.all_simple_paths(G, i, j)) for i in G.nodes_iter() for j in G.nodes_iter() if i != j and nx.has_path(G, i, j)]

rawDescription: Get list of parallel paths in a directed graph


code:
from functools import wraps
def count_check(function, count=[0]):
    """Returns number of times any function with this decorator is called
    """
    @wraps(function)
    def increase_count(*args, **kwargs):
        count[0] += 1
        return function(*args, **kwargs), count[0]

    return increase_count

rawDescription: Python - Count number of times function passes through decorator


code:
states = {}
for disease in seconddict['all_diseases']:
    state = disease.pop('status')
    states.setdefault(state, []).append(disease)

rawDescription: Merging 2 dicts with common elements in python


code: {x.pk:x for x in Blog.objects.all()}

rawDescription: how to get dict of model objects keyed by field

description: Get dict of model objects keyed by field


code:
x.close()
os.unlink(x.name)

rawDescription: Why does Python's tempfile method unlink() require a filename?


code:
from pyspark.sql.functions import col, rowNumber
from pyspark.sql.window import Window

w = Window.orderBy()
indexed = df.withColumn("index", rowNumber().over(w))

# Using DSL
indexed.where(col("index").isin(set(indexes)))

# Using SQL expression
indexed.where("index in ({0})".format(",".join(str(x) for x in indexes)))

rawDescription: PySpark DataFrames - way to enumerate without converting to Pandas?


code: [i+tuple(j) for i,j in d.items()]

rawDescription: Reduce list of lists if entries match

description: Reduce list of lists if entries match


code:
def ensure_finished(stream):
    try:
        unpack(stream, 'c')
    except struct.error:
        pass
    else:
        raise ValueError('Stream not consumed')
ensure_finished(b)

rawDescription: Idiomatic way to struct.unpack from BytesIO?


code:
class MyModel(models.Model):
    pass

class MyModelAgain(MyModel):
    class Meta:
        proxy = True

rawDescription: Django, multiple ModelAdmin for one Model


rawDescription: Writing camera matrix into xml/yaml file


rawDescription: Python uuid4, How to limit the length of Unique chars


code:
class Bar(list):
    def __len__(self):
        return sum(1 for ob in self if not isinstance(ob, Bar))

len(Bar([1, 2, 3]))
len(Bar([1, 2, 3, Bar()]))

rawDescription: Is it possible to modify the behavior of len()?


rawDescription: References to All Defined Objects in a Python Interactive Session


code: order_items = OrderItem.objects.all_or_filtered('provider', request.POST.get('provider'))

rawDescription: Django wildcard query

description: Django wildcard query


rawDescription: Sqlite executemany or union select for reduced db writes


code:
for sub in split_list(screws, 3):
    print([max(ele) for ele in sub])

rawDescription: local maximum in lists within list

description: Local maximum in lists within list


code:
def randint(a, b):
    "Return random integer in range [a, b], including both end points."
    return a + randbelow(b - a + 1)

def randbelow(n):
    "Return a random int in the range [0,n).  Raises ValueError if n<=0."
    # from Lib/random.py
    if n <= 0:
       raise ValueError
    k = n.bit_length()  # don't use (n-1) here because n can be 1
    r = getrandbits(k)          # 0 <= r < 2**k
    while r >= n: # avoid skew
        r = getrandbits(k)
    return r

rawDescription: Python Random Function without using random module


code:
#!/usr/bin/env python
import psyco; psyco.full()
from math import sqrt, ceil
import numpy as np

def rwh_primes(n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
    """ Returns  a list of primes < n """
    sieve = [True] * n
    for i in xrange(3,int(n**0.5)+1,2):
        if sieve[i]:
            sieve[i*i::2*i]=[False]*((n-i*i-1)/(2*i)+1)
    return [2] + [i for i in xrange(3,n,2) if sieve[i]]

def rwh_primes1(n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
    """ Returns  a list of primes < n """
    sieve = [True] * (n/2)
    for i in xrange(3,int(n**0.5)+1,2):
        if sieve[i/2]:
            sieve[i*i/2::i] = [False] * ((n-i*i-1)/(2*i)+1)
    return [2] + [2*i+1 for i in xrange(1,n/2) if sieve[i]]

def rwh_primes2(n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
    """ Input n>=6, Returns a list of primes, 2 <= p < n """
    correction = (n%6>1)
    n = {0:n,1:n-1,2:n+4,3:n+3,4:n+2,5:n+1}[n%6]
    sieve = [True] * (n/3)
    sieve[0] = False
    for i in xrange(int(n**0.5)/3+1):
      if sieve[i]:
        k=3*i+1|1
        sieve[      ((k*k)/3)      ::2*k]=[False]*((n/6-(k*k)/6-1)/k+1)
        sieve[(k*k+4*k-2*k*(i&1))/3::2*k]=[False]*((n/6-(k*k+4*k-2*k*(i&1))/6-1)/k+1)
    return [2,3] + [3*i+1|1 for i in xrange(1,n/3-correction) if sieve[i]]

def sieve_wheel_30(N):
    # http://zerovolt.com/?p=88
    ''' Returns a list of primes <= N using wheel criterion 2*3*5 = 30

Copyright 2009 by zerovolt.com
This code is free for non-commercial purposes, in which case you can just leave this comment as a credit for my work.
If you need this code for commercial purposes, please contact me by sending an email to: info [at] zerovolt [dot] com.'''
    __smallp = ( 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59,
    61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113, 127, 131, 137, 139,
    149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227,
    229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311,
    313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379, 383, 389, 397, 401,
    409, 419, 421, 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491,
    499, 503, 509, 521, 523, 541, 547, 557, 563, 569, 571, 577, 587, 593, 599,
    601, 607, 613, 617, 619, 631, 641, 643, 647, 653, 659, 661, 673, 677, 683,
    691, 701, 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773, 787, 797,
    809, 811, 821, 823, 827, 829, 839, 853, 857, 859, 863, 877, 881, 883, 887,
    907, 911, 919, 929, 937, 941, 947, 953, 967, 971, 977, 983, 991, 997)

    wheel = (2, 3, 5)
    const = 30
    if N < 2:
        return []
    if N <= const:
        pos = 0
        while __smallp[pos] <= N:
            pos += 1
        return list(__smallp[:pos])
    # make the offsets list
    offsets = (7, 11, 13, 17, 19, 23, 29, 1)
    # prepare the list
    p = [2, 3, 5]
    dim = 2 + N // const
    tk1  = [True] * dim
    tk7  = [True] * dim
    tk11 = [True] * dim
    tk13 = [True] * dim
    tk17 = [True] * dim
    tk19 = [True] * dim
    tk23 = [True] * dim
    tk29 = [True] * dim
    tk1[0] = False
    # help dictionary d
    # d[a , b] = c  ==> if I want to find the smallest useful multiple of (30*pos)+a
    # on tkc, then I need the index given by the product of [(30*pos)+a][(30*pos)+b]
    # in general. If b < a, I need [(30*pos)+a][(30*(pos+1))+b]
    d = {}
    for x in offsets:
        for y in offsets:
            res = (x*y) % const
            if res in offsets:
                d[(x, res)] = y
    # another help dictionary: gives tkx calling tmptk[x]
    tmptk = {1:tk1, 7:tk7, 11:tk11, 13:tk13, 17:tk17, 19:tk19, 23:tk23, 29:tk29}
    pos, prime, lastadded, stop = 0, 0, 0, int(ceil(sqrt(N)))
    # inner functions definition
    def del_mult(tk, start, step):
        for k in xrange(start, len(tk), step):
            tk[k] = False
    # end of inner functions definition
    cpos = const * pos
    while prime < stop:
        # 30k + 7
        if tk7[pos]:
            prime = cpos + 7
            p.append(prime)
            lastadded = 7
            for off in offsets:
                tmp = d[(7, off)]
                start = (pos + prime) if off == 7 else (prime * (const * (pos + 1 if tmp < 7 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 11
        if tk11[pos]:
            prime = cpos + 11
            p.append(prime)
            lastadded = 11
            for off in offsets:
                tmp = d[(11, off)]
                start = (pos + prime) if off == 11 else (prime * (const * (pos + 1 if tmp < 11 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 13
        if tk13[pos]:
            prime = cpos + 13
            p.append(prime)
            lastadded = 13
            for off in offsets:
                tmp = d[(13, off)]
                start = (pos + prime) if off == 13 else (prime * (const * (pos + 1 if tmp < 13 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 17
        if tk17[pos]:
            prime = cpos + 17
            p.append(prime)
            lastadded = 17
            for off in offsets:
                tmp = d[(17, off)]
                start = (pos + prime) if off == 17 else (prime * (const * (pos + 1 if tmp < 17 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 19
        if tk19[pos]:
            prime = cpos + 19
            p.append(prime)
            lastadded = 19
            for off in offsets:
                tmp = d[(19, off)]
                start = (pos + prime) if off == 19 else (prime * (const * (pos + 1 if tmp < 19 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 23
        if tk23[pos]:
            prime = cpos + 23
            p.append(prime)
            lastadded = 23
            for off in offsets:
                tmp = d[(23, off)]
                start = (pos + prime) if off == 23 else (prime * (const * (pos + 1 if tmp < 23 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # 30k + 29
        if tk29[pos]:
            prime = cpos + 29
            p.append(prime)
            lastadded = 29
            for off in offsets:
                tmp = d[(29, off)]
                start = (pos + prime) if off == 29 else (prime * (const * (pos + 1 if tmp < 29 else 0) + tmp) )//const
                del_mult(tmptk[off], start, prime)
        # now we go back to top tk1, so we need to increase pos by 1
        pos += 1
        cpos = const * pos
        # 30k + 1
        if tk1[pos]:
            prime = cpos + 1
            p.append(prime)
            lastadded = 1
            for off in offsets:
                tmp = d[(1, off)]
                start = (pos + prime) if off == 1 else (prime * (const * pos + tmp) )//const
                del_mult(tmptk[off], start, prime)
    # time to add remaining primes
    # if lastadded == 1, remove last element and start adding them from tk1
    # this way we don't need an "if" within the last while
    if lastadded == 1:
        p.pop()
    # now complete for every other possible prime
    while pos < len(tk1):
        cpos = const * pos
        if tk1[pos]: p.append(cpos + 1)
        if tk7[pos]: p.append(cpos + 7)
        if tk11[pos]: p.append(cpos + 11)
        if tk13[pos]: p.append(cpos + 13)
        if tk17[pos]: p.append(cpos + 17)
        if tk19[pos]: p.append(cpos + 19)
        if tk23[pos]: p.append(cpos + 23)
        if tk29[pos]: p.append(cpos + 29)
        pos += 1
    # remove exceeding if present
    pos = len(p) - 1
    while p[pos] > N:
        pos -= 1
    if pos < len(p) - 1:
        del p[pos+1:]
    # return p list
    return p

def sieveOfEratosthenes(n):
    """sieveOfEratosthenes(n): return the list of the primes < n."""
    # Code from: <dickinsm@gmail.com>, Nov 30 2006
    # http://groups.google.com/group/comp.lang.python/msg/f1f10ced88c68c2d
    if n <= 2:
        return []
    sieve = range(3, n, 2)
    top = len(sieve)
    for si in sieve:
        if si:
            bottom = (si*si - 3) // 2
            if bottom >= top:
                break
            sieve[bottom::si] = [0] * -((bottom - top) // si)
    return [2] + [el for el in sieve if el]

def sieveOfAtkin(end):
    """sieveOfAtkin(end): return a list of all the prime numbers <end
    using the Sieve of Atkin."""
    # Code by Steve Krenzel, <Sgk284@gmail.com>, improved
    # Code: https://web.archive.org/web/20080324064651/http://krenzel.info/?p=83
    # Info: http://en.wikipedia.org/wiki/Sieve_of_Atkin
    assert end > 0
    lng = ((end-1) // 2)
    sieve = [False] * (lng + 1)

    x_max, x2, xd = int(sqrt((end-1)/4.0)), 0, 4
    for xd in xrange(4, 8*x_max + 2, 8):
        x2 += xd
        y_max = int(sqrt(end-x2))
        n, n_diff = x2 + y_max*y_max, (y_max << 1) - 1
        if not (n & 1):
            n -= n_diff
            n_diff -= 2
        for d in xrange((n_diff - 1) << 1, -1, -8):
            m = n % 12
            if m == 1 or m == 5:
                m = n >> 1
                sieve[m] = not sieve[m]
            n -= d

    x_max, x2, xd = int(sqrt((end-1) / 3.0)), 0, 3
    for xd in xrange(3, 6 * x_max + 2, 6):
        x2 += xd
        y_max = int(sqrt(end-x2))
        n, n_diff = x2 + y_max*y_max, (y_max << 1) - 1
        if not(n & 1):
            n -= n_diff
            n_diff -= 2
        for d in xrange((n_diff - 1) << 1, -1, -8):
            if n % 12 == 7:
                m = n >> 1
                sieve[m] = not sieve[m]
            n -= d

    x_max, y_min, x2, xd = int((2 + sqrt(4-8*(1-end)))/4), -1, 0, 3
    for x in xrange(1, x_max + 1):
        x2 += xd
        xd += 6
        if x2 >= end: y_min = (((int(ceil(sqrt(x2 - end))) - 1) << 1) - 2) << 1
        n, n_diff = ((x*x + x) << 1) - 1, (((x-1) << 1) - 2) << 1
        for d in xrange(n_diff, y_min, -8):
            if n % 12 == 11:
                m = n >> 1
                sieve[m] = not sieve[m]
            n += d

    primes = [2, 3]
    if end <= 3:
        return primes[:max(0,end-2)]

    for n in xrange(5 >> 1, (int(sqrt(end))+1) >> 1):
        if sieve[n]:
            primes.append((n << 1) + 1)
            aux = (n << 1) + 1
            aux *= aux
            for k in xrange(aux, end, 2 * aux):
                sieve[k >> 1] = False

    s  = int(sqrt(end)) + 1
    if s  % 2 == 0:
        s += 1
    primes.extend([i for i in xrange(s, end, 2) if sieve[i >> 1]])

    return primes

def ambi_sieve_plain(n):
    s = range(3, n, 2)
    for m in xrange(3, int(n**0.5)+1, 2):
        if s[(m-3)/2]:
            for t in xrange((m*m-3)/2,(n>>1)-1,m):
                s[t]=0
    return [2]+[t for t in s if t>0]

def sundaram3(max_n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/2073279#2073279
    numbers = range(3, max_n+1, 2)
    half = (max_n)//2
    initial = 4

    for step in xrange(3, max_n+1, 2):
        for i in xrange(initial, half, step):
            numbers[i-1] = 0
        initial += 2*(step+1)

        if initial > half:
            return [2] + filter(None, numbers)

################################################################################
# Using Numpy:
def ambi_sieve(n):
    # http://tommih.blogspot.com/2009/04/fast-prime-number-generator.html
    s = np.arange(3, n, 2)
    for m in xrange(3, int(n ** 0.5)+1, 2):
        if s[(m-3)/2]:
            s[(m*m-3)/2::m]=0
    return np.r_[2, s[s>0]]

def primesfrom3to(n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
    """ Returns a array of primes, p < n """
    assert n>=2
    sieve = np.ones(n/2, dtype=np.bool)
    for i in xrange(3,int(n**0.5)+1,2):
        if sieve[i/2]:
            sieve[i*i/2::i] = False
    return np.r_[2, 2*np.nonzero(sieve)[0][1::]+1]

def primesfrom2to(n):
    # http://stackoverflow.com/questions/2068372/fastest-way-to-list-all-primes-below-n-in-python/3035188#3035188
    """ Input n>=6, Returns a array of primes, 2 <= p < n """
    sieve = np.ones(n/3 + (n%6==2), dtype=np.bool)
    sieve[0] = False
    for i in xrange(int(n**0.5)/3+1):
        if sieve[i]:
            k=3*i+1|1
            sieve[      ((k*k)/3)      ::2*k] = False
            sieve[(k*k+4*k-2*k*(i&1))/3::2*k] = False
    return np.r_[2,3,((3*np.nonzero(sieve)[0]+1)|1)]

if __name__=='__main__':
    import itertools
    import sys

    def test(f1,f2,num):
        print('Testing {f1} and {f2} return same results'.format(
            f1=f1.func_name,
            f2=f2.func_name))
        if not all([a==b for a,b in itertools.izip_longest(f1(num),f2(num))]):
            sys.exit("Error: %s(%s) != %s(%s)"%(f1.func_name,num,f2.func_name,num))

    n=1000000
    test(sieveOfAtkin,sieveOfEratosthenes,n)
    test(sieveOfAtkin,ambi_sieve,n)
    test(sieveOfAtkin,ambi_sieve_plain,n)
    test(sieveOfAtkin,sundaram3,n)
    test(sieveOfAtkin,sieve_wheel_30,n)
    test(sieveOfAtkin,primesfrom3to,n)
    test(sieveOfAtkin,primesfrom2to,n)
    test(sieveOfAtkin,rwh_primes,n)
    test(sieveOfAtkin,rwh_primes1,n)
    test(sieveOfAtkin,rwh_primes2,n)

rawDescription: Fastest way to list all primes below N

description: Fastest way to list all primes below N


code: now = datetime.datetime(2014, 3, 14, 12, 34)

rawDescription: Python - Convert time to different time zone with am/pm


code: x = np.linspace(-np.pi/4, np.pi/4, round(np.pi / 2 / 0.001) + 1)

rawDescription: Plotted arc: anomaly in the plot

description: Plotted arc: anomaly in the plot


code: isinstance(t, datetime.datetime)

rawDescription: about python datetime type

description: About python datetime type


rawDescription: The fastest way to find common elements at the beginning of 2 python lists?


code:
def newFrame(self, event):
    self.new_window = wx.Frame(self, title='frame2', size=(500, 500), pos=(800,0))

    self.scroll = wx.ScrolledWindow(self.new_window, -1)
    self.scroll.SetScrollbars(1, 1, 1600, 1400)
    self.new_window.Layout()
    self.new_window.Fit()
    self.new_window.Show()

rawDescription: creating a wxpython scrolled window (frame) by an event

description: Creating a wxpython scrolled window (frame) by an event


rawDescription: highlight single contour line

description: Highlight single contour line


code: from collections import defaultdict

rawDescription: Python dictionary to pairs and check for antisymmetry using dictionary


rawDescription: is there something like coo_matrix but for sparse vectors?

description: Is there something like coo_matrix but for sparse vectors


code:
HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
        'URL': 'http://127.0.0.1:9200/',
        'INDEX_NAME': 'haystack',
    },
    'robust_elasticsearch':{
        'ENGINE': 'YOURAPP.backend.RobustElasticSearchEngine',
        'URL': 'http://127.0.0.1:9200/',
        'INDEX_NAME': 'haystack',
    }
}

rawDescription: Catch ConnectionError with Django-Haystack and ElasticSearch


code:
from django.db import connection
connection.queries

rawDescription: django: is there a black magic on extra fields on many-to-many relationships?

description: Django: is there a black magic on extra fields on many-to-many relationships


code: new_new_list = new_list + random.sample(old_list, 5)

rawDescription: Choose 5 different elements from a list?

description: Choose 5 different elements from a list


code:
B.__class__ = MetaClass
MetaClass._actual_init(B, B.__name__, B.__bases__, B.__dict__)

rawDescription: Is there a way to set metaclass after the class definition?


code:
from nltk.corpus import cmudict # $ pip install nltk
# $ python -c "import nltk; nltk.download('cmudict')"

def spubeak(word, pronunciations=cmudict.dict()):
    istitle = word.istitle() # remember, to preserve titlecase
    w = word.lower() #note: ignore Unicode case-folding
    for syllables in pronunciations.get(w, []):
        parts = []
        for syl in syllables:
            if syl[:1] == syl[1:2]:
                syl = syl[1:] # remove duplicate
            isvowel = syl[-1].isdigit()
            # pronounce the word
            parts.append('ub'+syl[:-1] if isvowel else syl)
        result = ''.join(map(str.lower, parts))
        return result.title() if istitle else result
    return word # word not found in the dictionary

rawDescription: Python: How to prepend the string 'ub' to every pronounced vowel in a string?


code: id_find = int(input('Enter the unique student ID: '))

rawDescription: Referring to a dictionary using an input


rawDescription: Excluding web links with specific extensions in web scraper


code:
elapsed_time <= delta # Finished before (or on) the objective time
elapsed_time > delta # Finished after the objective time

rawDescription: How would I format inputted minutes/seconds into datetime.now() format?


code:
class Update_MyStore(MyStore):
    def __init__(self, store):
        db = shelve.open(store)
        for i in db:
            setattr(self, i, db[i])
        self.items()
        self.store_in_db()
Update_MyStore('store')

rawDescription: How to *append* a text to a database file opened with shelve?


code: data[x:x+2, y:y+2] = average

rawDescription: Shorter way to manipulate region of two-dimensional list


code:
class Post(models.Model):
    title = models.CharField(max_length=100)
    commenters = models.ManyToManyField(User)

me = User.objects.get(username='Josh')
p = Post(title="ManyToManyExample")
p.commenters.add(me) # error, Post does not yet have an ID.
p.save()
p.commenters.add(me) # success!

rawDescription: Triggering another model's data to change (M2M) based on this model


code: 'olleh'

rawDescription: Reverse a Python string without omitting start and end slice


code:
b.grab_set() # when you show the popup
# do stuff ...
b.grab_release() # to return to normal

rawDescription: Disable the underlying window when a popup is created in Python TKinter


rawDescription: How to turn a dataframe of categorical data into a dictionary

description: Turn a dataframe of categorical data into a dictionary


code:
slice(4,5).start
slice(4,5).stop
slice(4,5).step  #None

rawDescription: Parsing slice information from a slice object?


code:
def hasNonemptyDiff(A,B):
    foundExampleYet = False
    for a in A:
        if(a in B): # here's the change
            foundExampleYet = True
    return foundExampleYet

rawDescription: Python-New To Python and programming in general-difference of two sets


code:
def lookup(a_dict, key):
    for d in all_dicts(a_dict):
        if key in d:
            return d[key]

rawDescription: Find deeper in dictionary or list

description: Find deeper in dictionary or list


code:
from cust1.proj1.pack1.mod1 import something
from cust3.proj4.pack1.mod4 import something_else
from cust3.proj1.pack4.mod4 import yet_something_else

rawDescription: python import path: packages with the same name in different folders

description: Python import path: packages with the same name in different folders


rawDescription: Apply similar function to multilevel pandas dataframe


code:
import glob
import multiprocessing
from functools import partial

list_files = glob.glob("Protein/*.txt")

def Some_func(lock, some_file):
    with open(some_file) as some:
        with open(file_output) as output:
            for lines in Some:
                #Do Something
                #edited_lines = func(lines)
                with lock:
                    output.write(edited_lines)

pool = multiprocessing.Pool(10) # Desired number of threads = 10
m = multiprocessing.Manager()
lock = m.Lock()
pool.map(partial(Some_func, lock), list_files)
pool.close()
pool.join()

rawDescription: Writing Errors in Multiprocessing Python


rawDescription: Sorting child elements with lxml based on attribute value


rawDescription: Plotting the difference between two datetime64[ns]


code:
from mpl_toolkits.mplot3d import Axes3D
Znew = griddata( R,T,Z, Rnew,Tnew )
ax = Axes3D(fig)
ax.plot_surface( Rnew * np.cos(Tnew), Rnew * np.sin(Tnew), Znew )

rawDescription: Problem with 2D interpolation in SciPy, non-rectangular grid


code:
from collections import Counter
[k for k,v in Counter(['a', 'b', 'c', 'a', 'b', 'a', 'd']).most_common() for i in xrange(v)]

rawDescription: Sorting Lists by Repetitions in Python

description: Sorting Lists by Repetitions in Python


rawDescription: Issue with parsing html with lxml by xpath


code:
import re

tokens = (
  ('STRING', re.compile('"[^"]+"')),  # longest match
  ('ID', re.compile('[a-zA-Z_]+')),
  ('SPACE', re.compile('\s+')),
  ('DIGIT', re.compile('\d+')),
)

def tokenizer(s):
  i = 0
  lexeme = []
  while i < len(s):
    match = False
    for token, regex in tokens:
      result = regex.match(s, i)
      if result:
        lexeme.append((token, result.group(0)))
        i = result.end()
        match = True
        break
    if not match:
      raise Exception('lexical error at {0}'.format(i))
  return lexeme

rawDescription: Tokenizing User Input in Python

description: Tokenizing User Input in Python


code:
def inPoly(pt1, poly):

    points = poly.getPoints()
    nvert = len(points) #the number of vertices in the polygon

    #get x and y of pt1
    x = pt1.getX()
    y = pt1.getY()

    # I don't know why this works
    # See the link I provided for details
    result = False
    for i in range(nvert):

        # note: points[-1] will give you the last element
        # convenient!
        j = i - 1

        #get x and y of vertex at index i
        vix = points[i].getX()
        viy = points[i].getY()

        #get x and y of vertex at index j
        vjx = points[j].getX()
        vjy = points[j].getY()

        if (viy > y) != (vjy > y) and (x < (vjx - vix) * (y - viy) / (vjy - viy) + vix):
            result = not result

    return result

rawDescription: Checking if clicks are within a graphic object [Python Graphics Module]


rawDescription: In python the result of a /= 2.0 and a = a / 2.0 are not the same


code:
result = []
manufacturers = Manufacturer.objects.all() # get all manuf
for m in manufacturer:
    mf = dict()
    mf["manufacturer"] = m.name
    mf["models"] = m.model_set.all() # get all linked models
    result.append(mf)
# pass result to template as cars

rawDescription: Python: How can I implement this in a DRY way using Jinja2?


code: [l[i::5] for i in xrange(5)]

rawDescription: Displaying a list of items vertically in a table instead of horizonally


rawDescription: django object get two fields into a list from a model

description: Django object get two fields into a list from a model


code: old = x0

rawDescription: Using a stored value of a variable in a for loop


code:
a = np.random.randint(100, size=(5,5))
c = np.random.randint(100, size=(5,5))
a
c
neighbor_min(a, c)

rawDescription: Numpy: fast calculations considering items' neighbors and their position inside the array


code:
# Again, open the three files as f1, f2 and outf
with open('new1.txt') as f1, open('new2.txt') as f2, open('diff.txt', 'w') as outf:

    # Read the lines in 'new1.txt' and 'new2.txt'
    s1, s2 = set(f1.readlines()), set(f2.readlines())

    # `s1 - s2 | s2 - s2` returns the differences between two sets
    # Now we simply loop through the different lines
    for line in s1 - s2 | s2 - s1:

        # And output all the different lines
        outf.write(line)

rawDescription: How to compare 2 txt files in Python

description: Compare 2 txt files in Python


code:
from copy import deepcopy
for element in elements:
    table.append(deepcopy(element))

rawDescription: Taking data from an XML document and writing it to another XML document with python


code:
def chunk_sequences(it, n):
    """
    Yield all sequences of n from iterable.
    """
    chunk = []
    for x in it:
        if x == n:
            chunk.append(n)
        else:
            if len(chunk) > 0:
                yield chunk
                chunk = []
    if len(chunk) > 0:
        #needed in case the last sequence runs into the last element
        yield chunk

rawDescription: Python: How to find sequences of readings with same id and calculating the interval between first and the last element for every such sequence


code:
s = '''
{
    orange: '2',
    apple: '1',
    lemon: '3'
}'''
import yaml
yaml.load(s)

rawDescription: Python: How can I parse { apple: "1" , orange: "2" } into Dictionary?


rawDescription: How to convert date string to ISO8601 standard?

description: Convert date string to ISO8601 standard


code: duplicates = models.ManyToManyField('self', null=True)

rawDescription: Relate image model with similar images

description: Relate image model with similar images


code:
class Event(Base):

    #tablename
    type = Column(Unicode(50), nullable=False, index=True)
    #foreign id
    table_id = Column(Integer, nullable=False, index=True)
    attr = Column(Unicode(50), nullable=False)
    old = Column(Unicode(50), nullable=False)
    new = Column(Unicode(50), nullable=False)

class MetadataMixin(object):
    @declared_attr
    def events(cls):
        joinstr = '{0}.id == foreign(Event.table_id) and Event.type == "{1}"'.format(cls.__name__, cls.__tablename__)
        return relationship('Event', primaryjoin=joinstr, backref=cls.__name__.lower())

rawDescription: Many to Many with Multiple Parents in SQLAlchemy


code: self.msg.content_subtype = "html"

rawDescription: Django EmailMultiAlternatives and HTML e-mail display in Outlook 2003 on Win2003


code:
import sys
sys.stdout = OutLog( edit, sys.stdout)
sys.stderr = OutLog( edit, sys.stderr, QtGui.QColor(255,0,0) )

rawDescription: PySide and python logging

description: PySide and python logging


rawDescription: What is the fastest way to slice a scipy.sparse matrix?


rawDescription: How can I iterate through the indices of the list?

description: Iterate through the indices of the list


code:
import matplotlib.pyplot as plt
plt.figtext(.5,.9,'Temperature', fontsize=100, ha='center')
plt.figtext(.5,.8,'Humidity',fontsize=30,ha='center')
plt.show()

rawDescription: How to increase plt.title font size (python)

description: Increase plt.title font size (python)


code:
def next_player(self):
    i = players.index(self)
    return players[(i+1) % len(players)]

rawDescription: How to call functions of next/previous object in list in Python

description: Call functions of next/previous object in list in Python


code:
class MyFighters(Fighters):
        def foo_func(self):
                print("making mybar")
                self.x = MyBar()

class MyBar(Bar):
        def bar_func(self):
                print("mybar bar_func")

rawDescription: Python: multi level inheritance

description: Python: multi level inheritance


code:
bad = sizes.index[sizes < df1.shape[0]*0.1]
bad

rawDescription: Reassigning strings by value count in Pandas