Remember Python - Memory Profiling in Python

67 %
33 %
Information about Remember Python - Memory Profiling in Python

Published on March 12, 2014

Author: baraksch


Python memory profiling Barak Schiller

Your team wrote a python program to do station recommendations You run it After 10 min you machine grinds to a halt and a cryptic “out of memory” appears You go and make some coffee/tea So...

Memory management in python Once upon a time there were only references But then the reference cycles appeared and multiplied And then there was gc, and Guido saw that it was good

Code time!

More on GC Two triggers: manual and on allocation threshold Internally tracks all containers and looks for cycles Common wisdom is to run it once after your initialisation phase, then every so often Avoid __del__! (same as in Java)

memory_profiler module Adds a @profile annotate Uses psutil to track memory consumption Covers C-side allocations Supports plotting memory usage over time

Code time! Now actually running!

objgraph Generates object dependency graphs Useful when you want to crawl a specific object

Good old printf debugging What I ended up using was… printf! Wire up psutil into the python-core logger Get a basic memory usage throughout the application lifetime

Add a comment

Related presentations

Related pages

How do I profile memory usage in Python? - Stack Overflow

How do I profile memory usage in Python? ... I'm already familiar with the standard Python module for profiling runtime ... Python memory profiler.
Read more

Issue 2898: Add memory footprint query - Python tracker

Remember me? Register; Lost your login? Administration. User List; ... This patch is part of my Google Summer of Code project on Python memory profiling ...
Read more

Issue 3329: API for setting the memory allocator ... - Python

Profiling email: py _setallocators ... Currently Python always uses the C library malloc/realloc/free as the underlying mechanism for requesting memory ...
Read more

How to write a memory efficient Python program? - Stack ...

How to write a memory efficient Python program? ... Try profiling memory ... as you are no longer creating as many Python values. Remember that a variable ...
Read more

profiling memory usage - Python

profiling memory usage. Python Forums on ... home > topics > python > questions > profiling memory usage ... Remember that time Thotsakan punched Neil ...
Read more

High Performance Python: Profiling -

High Performance Python: Profiling ... This article is part of an ongoing series about profiling Python programs ... execution time and memory ...
Read more

Profiling · evennia/evennia Wiki · GitHub

evennia - Python MUD/MUX/MUSH/MU* development ... evennia / evennia. Code. Issues 24. Pull requests 2. Wiki Pulse Graphs Profiling. Griatch edited this ...
Read more