@2021/11/30

I really never thought that it is necessary to explain why I use PyCharm, but the discussion on my previous blog Tools I use: PyCharm / Idea plugins made me see, that it is.

IDE

Let's start with the obvious thing, which is not so well understood, as I found out when I was talking about it with dozens of people:

👉

PyCharm is an IDE. That's a good thing.

In the past, I used to have weird misconceptions about what IDE really means. The acronym of course means Integrated Development Environment. I could understand the words, but could I really understand the experience? I mean, without getting traumatic flashbacks back to the college, when I was forced to use large and clumsy things like Eclipse or NetBeans?

Here is what I thought the acronym means:

1. Integrated → All kind of features in one program, and you can use buttons to click on things, instead of running custom commands like git gui in the background.

2. Development → It has features important for developers, like debugger and programmer's editor.

3. Environment → The program has some kind of framework for everything. You can usually extend it with plugins.

And while all of this is certainly true to some degree, it is also a misconception, which I'll address later.

The other approach to IDE

Just to be fair, let me say something about the other approach to IDE, which is to build your own customized pseudo-IDE from extendable editors.

Historically, I've used Vim for several years as my main editor, and I still use it on server and in terminal when needed. Other than that, I have tried all kinds of other editors and IDE's:

Turbopascal, DeD, PSPad, Ulipad, SciTE, jEdit, Gedit, Geany, Kate, WingIDE, Netbeans, Eclipse, Visual Studio (for C#), Monodevelop and (Space)Emacs. All of them for at least couple of months, if not years. Since 2013 to 2018 I've been using Sublime text, which I really loved.

In 2018, I switched to VS Code, mostly because I've changed my job, and I was working on a big code base. Something like 60 thousand sloc (just the lines of code, without comments) lines of Python code in one package.

I've worked on larger code bases before, but usually, they were split into smaller sub-packages. Now, I needed something which can analyze the whole code base and offer me type hinting and autocomplete. And VS Code could do that, and more.

I could quickly install all the plugins I required. Debugger, code refactoring, even code understanding in the form of Microsoft Python Plugin, and Pylance, which can do more advanced analysis, than what I was used to before. I was pretty happy. Mostly.

I mean, it mostly worked. From time to time, something broke. Refactoring suddenly stopped working after an update. Code analysis would eat up 20 gigabytes of memory, and needed to be restarted periodically. I even wrote a script, which would run in the background and kill VS Code language servers:

kill_memory_eaters.py

But in general, I wasn't complaining.

My approach changed one day, when I was in a pub with the colleagues from my previous job, and we were talking about the editors we are using. And most of the guys there told the same story:

I use PyCharm, but it's annoyingly memory hungry. It takes like 2 gigabytes of RAM!

I was like .. wtf? 2 gigs? My VS Code usually takes 16 and spikes so much that it kills my whole system from time to time. That was the first push.

The second push came when I was visiting lectures about GraalVM in a local university, and the teacher would use high-level refactorings without even thinking. All kinds of stuff like "make a method from a piece of code", or "move this here and there".

I've realized, that while I was able to do the same, I was never quite sure it would work. Typically, VS Code plugins could do the refactoring I had in mind, but fuck up various parts of the process. Like forget to move imports. Or overlook half of the variables used in the code and then require me to rewrite the newly created method so that it would accept them as parameters.

So, I gave PyCharm a try. But I started, quite untraditionally, by RTFM. I've got the Effective Pycharm book, and I've read it from cover to cover, taking notes and making highlights on what I wanted to use and configure.

I won't lie — it wasn't an easy process. I have a huge number of obscure shortcuts I've been using in previous editors, like really arcane stuff, that I have no idea where I've picked up. Likewise, I had to create a few macros, and configure several plugins. But after a while, it worked. And I started to enjoy it.

Integrated

Now, here is the part that I didn't really get before I started using PyCharm:

👉

Integrated means different things.

All of which PyCharm got right. It doesn't just mean that things are already there. That, for example, debugger comes packaged with the editor, and can be readily used. It also means:

Someone else does the integration

The obvious but crucial point is that someone else does the integration. That means that someone else created it, made it work together, and there are probably testers that test all the required functionality before release.

This is a huge difference from the "build your own IDE" approach, where things break all the time because someone, who does a voluntary work, released a buggy version of the plugin. Or the plugin conflicts with your other plugins. For example, PEP8 plugin clashes with Pylint.

In IDE, things just work all the time, and someone else does the job required to keep them working. This directly saves your time.

It is designed to be what it is

Because the IDE is a conscious effort, there are people who make the decisions about the development so that it is most beneficial for the user. People actually think about architecture, about the features, about the graphical interface.

While in "build your own IDE" approach this is usually true for the editor itself, the rest is just a collection of plugins slapped together on the skeleton of the editor. One plugin prints the results to the console, a second shows a popup, and a third opens a new tab in the editor, where it fills the new tab with some kind of text. Good luck trying to make them all work together.

People who make PyCharm realize that things are not standalone. That debugger needs to communicate with the IDE, and the editor, console and multiple other things, like live inspection of objects in memory in a custom graphical interface, which is also part of the IDE.

Things are consistent

Because it is designed to be what it is, things that work in one place work in other places. I can for example still use all of my standard editor features (shortcuts, multi-cursors and so on) when I open the commit dialog and write a commit message. Or when I am in the inspector doing debugging.

Everything I need is integrated

I don't think this needs to be explained, but really, this feels great. The program comes with everything. And all the tools talk to each other.

For example, when you work with databases in Python code, it will ask for the data source so that it can autocomplete your SQL queries, and check them for bugs against the structure of the database itself. But there are many other usages, like debugger showing you the content of the variables in the source code editor, instead of just the debug sidebar. And so on.

There are great abstractions

This is more about the "Environment" part of the IDE, but what the people behind PyCharm did is really admirable: they built abstraction for things like versioning, or databases.

What it means is that it doesn't matter what kind of database you are using, or if you are using git or mercurial. The IDE will feel the same. These things are not some random plugins, but deep parts of the IDE. If it doesn't support the database you are using, you install a plugin that will tell the database manager how to work with it. You don't have to install a plugin that will add a new mercurial dialog, which will feel and work completely differently.

What is in the environment

Let me just quickly skim trough what I use in PyCharm on a day-to-day basis. Note that I am using the paid version, some features may not be enabled in the community version.

And everything works out of the box, without any need to fiddle with twenty plugins and their configuration. Of course, there is a lot more functionality I've probably forgotten, or not use so often. For example, great Kubernetes integration, so you can manage containers or even whole clusters from your IDE, or similar thing for Docker.

I won't go over all features I've shown in the image above, just some highlights of things I really like:

Superior understanding of the code

What you can't see from the screenshots is that PyCharm has probably the most advanced understanding of the code, I've ever seen. After all, JetBrains have to put the money somewhere.

PyCharm does a lot of type inference, error checking, suggests you more optimal ways how to write your code, tells you that you are using antipatterns, or that the cognitive load of your method is too big. Integration with linters literally makes you a better programmer.

This is the biggest reason, why refactoring and all other features work so nicely. It can also do all kinds of analysis, like to find methods which do the same things, and can be thus abstracted to some shared base-class, or show you the UML diagram of your code:

Local history

PyCharm keeps track of your files every time you do some kind of change. Think of it as a git, which works every time you save the file. You know that feeling, when you do 20 undo steps just to copy something you've already deleted, and then 20 redo steps, hoping that you won't type something accidentally? You don't have to have that feeling, ever, when using PyCharm.

Git integration

Speaking of git — I really love the integration. I know that pretty much any IDE has it, but this is the first IDE, where I actually like it. I used to have git gui and gitk opened on the second monitor all the time, and sometimes do things from the terminal. Not anymore.

I use it as a sidebar, so I can collapse it at any time using a simple shortcut. But you can, of course, configure it however you like.

You can also create pull requests, with preview and whole pull request management sidebar, if you are using GitHub or other supported sites. Think about it. I can make a pull request, assign it to my colleague, interact with him, and then merge the pull request on GitHub, without ever leaving my IDE.

Semantic search & internal command prompt

Semantic search is great. I can just press shift twice, and immediately search in the class names. Or in the symbol names. Or in the filenames. There of course is also a standard string search through the entire project, if that is not enough.

And I have ctrl+shift+p (old Sublime shortcut) mapped to Editor actions search, so I can invoke any action from GUI with fuzzy search. So for example, ctrl+shift+p push, and I am pushing my changes. It remembers my last commands, and is, in general, very nice.

Database view

Ah, I love database view so much. You can, of course, run SQL queries, look at the results shown as tables, have autocomplete and type checking in your Python code where you run your queries. But you can also explore the databases, look at the tables, even see the representation of the database as diagrams:

And of course, you can do modifications, like adding the tables and so on.

Final words

If you are not convinced, that's fine. I was really mostly explaining why I use it, to people who were curious and asked me.

For me, what makes the most significant difference is that .. PyCharm doesn't sux. Which is a weird thing to say, I know. But it basically never makes me angry, doesn't annoy me, works as expected, if not twenty steps ahead of my expectations. I like it. Which, for me, is extremely rare. I can tell you twenty disadvantages of literally every software I use, even the software I wrote for myself, completely custom, to fulfill my needs. And I can't tell you any real disadvantages of PyCharm.

Maybe I wish you could write plugins in Python, and not just in Java. But I don't really have any plugin I want to write. Maybe that I would like if it would be embeddable into other software, so I could use it as an embedded editor in my projects. But almost nothing is embeddable in this way these days, so it's not really fair.

If I should really sum my philosophy about tools of the trade:

Life is too short to use tools which are not as effective, as they can be. Compound interest has a considerable effect in the long run. The sooner you'll get better, the more you'll get better.

Really, my only regret is that I have not decided to use PyCharm sooner. I've learned, the hard way, that the more your IDE understands your code, the more effective you'll be. Especially for refactoring. Quick, effortless refactoring allows quick iterations and improvements. Which make everything better.

Notes

1. I am using TTF version of the Terminus font, but the original JetBrains font is also good.

2. And also 🗒Sunburst for PyCharm / Idea color scheme.

Note on Fleet

*Sigh*. JetBrains just released information about Fleet, which is apparently the new future. I am trying to be open-minded, but from what I can see, it has a fraction of the features I use on day-to-day basis, especially on the GUI side. But it is just a preview, so we'll see.

Note on rewrites

I am 32 years old, and during my life, I don't think I ever appreciated an upgrade to a new look, or new paradigm. For some reason, I can only remember Fallout 1 & Fallout 2, and that used the same engine. Literally, I've begun using Linux, after MS fucked up with Vista. I started using Palemoon & Chromium, after Firefox changed GUI for the third or fourth time, and it took me 7 years to even give it a second chance. And I could go on.

The problem with new looks is usually deeper than a change in the look and feel, although that is annoying as well. It lays in what K Lars Lohn in this beautiful video calls fractal dimension, which he uses to measure complexity:

To explain, let me borrow this image from wikipedia:

Fractal dimension can be used to measure the rate between how much curvature (or divisions) you have in the object itself, and the border you draw around it. If you want to measure the distance of the coastline, you quickly discover, that the problem gets harder the more precise you are trying to be. You can quickly get absurd numbers, which is explained in Coastline paradox.

This has a direct analogy in the software world, where you can in principle measure the rate between how much is your implementation modeling and describing the problem you are trying to solve.

If you ever worked in some kind of user form validation, you know what I am talking about. For example, let's say, that you want to validate user registration for a simple e-shop. Okay, so you make some inputs mandatory, like name and address. Later comes your boss, and tells you that you have to validate the address, that it really exists. Then validate the phone numbers. Then there are special rules if the customer is buying the stuff for themselves, or as a company, in which cases they don't have to pay some taxes. Maybe enable some inputs, and hide others if this or that.

After months of development, your form seemingly looks the same, but there is a huge code base with all kinds of validation and hidden logic. The fractal complexity increased. Your code is now curved like a fractal, countless ifs and method calls. The reason is, that you've traced the coastline of the problem space with your code more closely.

Now, a good IDE has a huge number of fractal complexity hidden in seemingly simple user interface. The problem with this, is that people who are doing "complete rewrite" usually look at this, say "what a mess" and throw it all away, just to painfully reinvent it in the years to come because the coastline of the problem space is still the same.

I know that I am probably more pessimistic than necessary. And there are certainly ways how to capture the fractal complexity of the problem coastline cleanly, and how to abstract things so that it's all nice and neat. And maybe they can rewrite it all without sweating. But I have yet to see a rewrite which wouldn't mess up most of the hidden complexity. Maybe I'll be surprised by this one.