Skip to content

Instantly share code, notes, and snippets.

@NostraDavid
Created September 19, 2024 09:56
Show Gist options
  • Save NostraDavid/12b68b78256d8fcd78e0b6ec519cd82b to your computer and use it in GitHub Desktop.
Save NostraDavid/12b68b78256d8fcd78e0b6ec519cd82b to your computer and use it in GitHub Desktop.
Pythonic Data Engineer (GitHub)

Pythonic Data Engineering Style Guide

The Essence of Style

Our coding style is a collective effort, blending engineering and art, aiming for precision and creativity. The best is yet to come.

Why Have Style?

Good style advances our design goals: readability, maintainability, performance, and developer experience.

On Simplicity and Elegance

Simplicity unites our design goals, achieving elegance by solving multiple concerns simultaneously.

Technical Debt

We maintain a "zero technical debt" policy, aiming to do it right the first time.

Safety

  • Employ simple control flow and minimal abstractions.
  • Limit everything; prevent infinite loops and spikes.
  • Use explicit types.
  • Assert preconditions, postconditions, and invariants.
  • Handle all errors.
  • Always explain your rationale.

Performance

  • Consider performance early.
  • Sketch resource usage.
  • Optimize bottlenecks.
  • Batch operations.
  • Be explicit in your code.

Developer Experience

Naming Things

  • Select precise names.
  • Use snake_case; follow PEP 8.
  • Include units in names.
  • Write informative commit messages.
  • Use comments to explain 'why' and 'how'.

Cache Invalidation

  • Avoid unnecessary variable duplication.
  • Use immutable structures.
  • Build objects in-place.
  • Limit variable scope.
  • Simplify functions.

Off-By-One Errors

  • Be cautious with indexes and counts.
  • Include units in names.
  • Clarify division operations.

Style By The Numbers

  • Use code formatters like black.
  • Indent with 4 spaces.
  • Keep lines under 79 characters.

Dependencies

  • Limit dependencies to essentials.

Tooling

  • Standardize tooling with Python.
  • Write cross-platform scripts in Python.

The Last Stage

Keep experimenting, have fun, and strive for elegance and efficiency.

“You don’t really suppose, do you, that all your adventures and escapes were managed by mere luck, just for your sole benefit? You are a very fine person, Mr. Baggins, and I am very fond of you; but you are only quite a little fellow in a wide world after all!”

“Thank goodness!” said Bilbo laughing, and handed him the tobacco-jar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment