My Journey in a Year

Category

Ben's Thoughts

Tags

ide, python, regex, regular expressions, technology, udemy, year

Posted

August 03, 2021

Now that I finished what was the last in my essential portfolio projects, I feel like it’s time to explain everything. I somewhat, sorta did, way back a few months ago. But I didn’t really examine things, just give a brief summary of things. So I’ll try to explain myself, my thoughts and what’s been happening.

Let’s start back in mid-August of 2020, virtually a year ago. By the way, that’s not a coincidence. I’ve been busting my butt to make sure that I could write something like this post around 1 year after I started. I learned that my wife was pregnant with my second son, and I decided then and there that I had to make a career change. Writing is by far the best thing that I have ever done. I learned so much about the world and myself through writing, and I’d like to think that my writing reflects my abilities. I still love it much more than programming, but it’s hard to make money with that. Scratch that, actually. It’s hard for me to make money with it. Some people make a really good living off of it. But you have to be good at making money and have an eye for marketing and marketability, more so than any sort of art or expression. I’m not speaking ill of those who do; what I’m saying is that I’m too stubborn and unwilling to do that sort of thing.

Back to programming. I had gone to community college somewhere around 2000 and learned some Visual Basic. I also learned some C++ and Java but stopped not long afterwards. To explain why, until I started to go to college, I always pursued the path of least resistance. I was really good at Algebra so I always liked math class. Then when I started learning limits and intro to calculus, the fact that I had to learn something new, and that meant effort. So I did really bad that semester, until it started to click. I realized at a certain point that I wouldn’t be able to advance much with programming unless I really put in some effort, and so I kinda just dropped out.

But now, with the patience, experience and ability to learn that comes with being 30-years-old, I decided I could do it. I knew one thing though: I loved writing, and I had already studied for a long time. I didn’t want to do anything that would require me to learn entirely new concepts and ways of looking at things if I didn’t have to. But I had no idea of anything at all. But I had heard about being able to take classes online for free and specifically, the Python programming language.

Thankfully, programming is much more beginner-friendly now than it was before, when I started looking up info about free sources online. I got started with Google’s python class. If you are thinking of getting into programming, I would most definitely not suggest it. Not only is it really old information (though it still works), it makes an assumption that you know what you’re doing with your computer, its file systems and the terminal. The terminal was something I only ever faintly understood before, even in those before times. For example, I compiled C++ and Java through something that was just like Notepad but more advanced and for programming. I didn’t even know the term for it! The word, I later learned, is IDE. And that’s an important part of it.

If you look on that web page for getting set up, it suggests you download something called TextWrangler or JEdit. TextWrangler has become BBEdit, and that works alright. I got it to work, and it has a nice feature to run the python program in the IDE, making it so you didn’t have to use the terminal. I, of course, took this opportunity to not learn how to use the terminal. If you look through the exercises, they often use line arguments (basically when you run the program, you don’t say ‘python -m program.py’ but ‘python -m program.py 1 2 3’, which I couldn’t do at all, either conceptually or in BBEdit. I just didn’t know how or where my files were and how the command line really worked. So I had to go through each program and make it work without arguments. You might think it’s extra work so it evened out. I guess, but I didn’t know how to use the terminal at all, and the changes weren’t very difficult to do. It doesn’t require a genius to lobotomize a python script.

I struggled through the lessons, and I learned how to do some stuff. Most if it wouldn’t really sink in until later, but I learned a decent amount, mostly with manipulating arrays, dictionaries and a little of regular expressions. But, at the end of it, I still didn’t know how things were really working. I just got things to work and didn’t worry about it too much.

Then I dived into two books. Because that’s how I remembered learning the last time, twenty (!) years ago.

So, now I’ll talk about IDEs as a concept, now that I get them. If you ever want to get into programming, it’s one of the most basic things and fundamental that you’ll need. It’s exactly what I said: basically notepad. But an IDE is to programming what Microsoft word is to writing things, or if you’re into good text editors or good IDEs, you can find something that’s the ProWritingAid of programming. It helps you organize things will capture things like what parts of speech you’re using and when you make a mistake (like the red squiggly bar). For example:

There’s an error!
What’s the error?

Don’t worry about what that error means. I’ll explain it, but you really don’t need to know. My point is just to show that my IDE is keeping track of what errors there are in relationship to my file directory and what modules are available.

Moving on, I had to learn about these, to get to the above point. I went through about several IDEs before I came to Atom, which seemed to be pretty awesome. It was for awhile, but then I found VS Code, and my mind really started to open up. This was around, November, I think. So I went through the first three months without a good grasp on exactly what I was doing or why. I felt pretty adrift for awhile with how programming languages worked, packages and the rest. It is really one of the biggest hurdles, to know how to structure your files, where things go and where things don’t go, how the programming language itself works and why this information is important.

To go back to those books, I want to name them. Because the ones I read ranged from decent to good. There are a ton of good resources on the internet, and you can find some of them at: https://pythonbooks.org/free-books/. The Think Python book taught me some stuff that no one else really taught, which I really, really appreciated (namely testing and some utility stuff that makes handling lists more efficient). The only caveat is that the book’s really for someone who already knows a bit of Python, so make yourself comfortable with what you’re doing before you get into it.

So, the next step (the one that really got me going in the right direction) was when I started buying classes on Udemy. The first purchase was an impulse decision because it required less commitment than EDX/Coursera, and I thought I was done with university classes. There was a sale on, and some of the classes were available for $10 instead of the usual $100-150. Well, little did I know, but there are sales on for usually about half the days in the year.

There are some bad classes that won’t really help you on Udemy. They will help people that already know what they’re doing, but at that point, you can just read the docs, and they’re not that much easier. The good classes are beginner classes because they’ll discuss what your development environment needs to be (which can be useful because sometimes it isn’t as obvious as the docs make it out to be), and you can skip ahead to what you need.

However, let me recommend, out of the box, some good classes. Colt Steele teaches a bunch of classes that are great. The classes slow walk you through a bunch of concepts that get incrementally more difficult but without overwhelming you (at least me). His web developer bootcamp is where I got started with HTML and then JavaScript. It is a great place to start. Then anything by Maximillian Schwarzmüller is great, especially when you have the fundamentals down (though he has his own fundamentals class) because he often makes assumptions that you know how to do certain things. It’s how I got my started with Angular/React/Vue/Svelte. Then others I would recommend would be this great class that finally got me to stop hating CSS and learn how to work with it. Others I would recommend is pretty much anything by Angela Yu (I only took the iOS class), Stephen Grider (a bit better for newbies, he slow walks things. However, from the other side, he sometimes brushes over something complicated without explaining it, which is why I don’t like him as much, though he teaches a lot of cool things). Then one last recommendation is this WordPress class. It’s the one class I took on WordPress, which basically enabled me to make everything on benyakiredits.com. It doesn’t go into a ton of detail about some important systems and design patterns, such as PHP or a lot of WordPress features, but it’ll teach you enough to do most of what you want to do.

Just to tell you, between the classes and what I did with them, that leads me to today. By now, I can just read the docs on a website to learn it and look at examples to learn it. What’s holding me back from learning more right now is a lack of will and a lack of time because, if there’s anything else to know about programming, it’s incredibly tedious and doesn’t tax your mental resources. It taxes your patience and ability to think creatively in very constrained ways.

There’s one important caveat about the classes. They require a lot of self discipline to actually learn from because no one is holding your hand for assignments. Some teachers have them, but you can just skip over them, and they don’t require a ton of attention. Nor are they particularly good at hammering home certain concepts, at least for me. Usually, I just wanted to get them out of the way, so I didn’t think too heavily about them. What I found useful is that after I was done a class, I would make a project from the ground up that uses that knowledge.

I actually have a list of technologies I have used and still want to use. If you want to see it, off the top of my head, these are the things I’ve used:

  • Python
  • Flask
  • Django
  • JavaScript (plain)
  • Vue
  • Angular
  • React
  • Svelte
  • TypeScript (used in many of these projects)
  • Apollo/GraphQL
  • GatsbyJS
  • SvelteKit
  • PHP/WordPress

And what I still have to use:

  • Next
  • Nuxt
  • Flutter
  • Golang
  • C#
  • Unity
  • Swift

I heard once that one of the most frightening things to a programmer is an empty project. It was true, that creating a new folder and saying “well, what now?” is often the worst thing. Now I know what to do. Usually, I plan out, at least mentally what I want to do and how to get there, at least the big strokes.

Let’s take my first project, r2api. My goal was simple: look at recipes on the web then translate them. So what did I need to do? First, I would need to access the website. How do I do that? The easiest way to access a website then use the data is web scraping. The main library to do this is Beautiful Soup in Python (which was the primary language I was using at the time, and I’d still use Python for anything that doesn’t need a website). Then we take that information and look for the precise information we need. So involved me looking at the website, looking at the code, and then trying to figure out ways to systematically find the information I was looking for. The thing is, most recipe websites are extremely consistent because they use templates. This means that every recipe on one website will be for example “title is the first thing you see, then story paragraph, then list of ingredients then recipe steps”. So this meant that I could go for those things.

Now, I have the information, and what I do with it is process it so it’s in an easier to handle format. That’s just a matter of normal logic. I’ll tell you that it involves a lot of loops and regular expressions, which is the worst thing if you don’t already know what they are. But they are incredibly necessary if you’re ever going to handle any sort of text.

Since this is my blog, and you obviously have read this far along, I’ll explain. Regular expressions is when you have to get specific letter and/or combinations from a string. For example, let’s say I want to take any paragraph then make sure that it only goes for the first 200 letters then I write … to signal that this is a portion of the text. For example, to give the excerpt of something. That’s easy, I just grab a slice from letter 1 (index of 0) to letter 200 (index 199). But what if I want it so that it goes back to the last complete word then writes the … so a sentence doesn’t get cut off as “and then he said mo…” but rather “and then he said…” and make is so every time it goes only the last space. Well then you’d want to look at the last space then the word after then remove them both.

So there are ways to do this without regular expressions, but they’re a bit more onerous. Regular expressions, instead, have this great feature where I say, explicitly, “last space then anything after gets replaced by …” The regular expression I used was something like this:

const sentence = 'so then he said more'
sentence.replace(/\s\S+$/, '')

This is JavaScript, which requires you to wrap regular expressions inside of a pair of //, so the actual regular expression is \s\S+$ — which I’ll explain now. \s means space, \S means anything but a space, and $ means the end of the sentence. So what this regular expression does is select the last space then any other non space character at the end of a word. Then the way I used it above is that I said, whenever you see this pattern (only at the end of a sentence), replace it with ”, which means just nothing because strings are wrapped with quotation marks.

You can do a whole lot of complicated things with regular expressions, basically anything at all you want with a sentence. However, they are hard to read. The one good thing is that they’ve been around forever, so pretty much any regular expression you could want is floating around on the internet. For example, email addresses follow a strict pattern of what is and what isn’t allowed. Do I know the rules? No way. But there are conventions and ruling bodies that decide the rules, and they put out RFC 753 (I think), and there’s a regular expression to go with it.

/^(([^<>()\[\]\\.,;:\s@"]+(\.[^<>()\[\]\\.,;:\s@"]+)*)|(".+"))@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}])|(([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/

I can parse what that means offhand, but the exact details can be a bit annoying and time consuming to grasp. But it’s helpful when you’re asking for an email address, and you don’t want to get something like “I don’t know”. Okay, here’s one I use in r2api for recipes from Giallo Zafferanno.

(\(.*\)|\D*)?(\d*[,\./]?[\d]*)[\s]*(\D*)(\d+)?[\s]*(\D+)?

So the last question is, what do I want the experience to be like? Nobody’s ever going to use my project, so there won’t be a community of people out there to help anyone who wants to use my package. I like the GatsbyJS approach of batteries included but removable. You can use it in the default form, which is basically to choose the appropriate converter (these are each custom tailored to each website) then plug the website in, and you get your parsed results back. Or you can do a bunch of custom stuff like just parse a website’s ingredients or translate the recipe.

So I want to get back to what I did in a year. Effectively, I took the classes from October until March then I worked on projects. I’ve taken very few breaks from programming (mostly to write), but I’ve nearly completed the projects I wanted to. I thought I had finished about a month ago, but then I started working on benyakir-writes.com. I’ve learned so much, and I’m proud of what I’ve accomplished while still writing. I remade my publisher’s website, I’ve made two versions of my blog, I made a website for a fake hospital as advertising for one of my books, and I made a fullstack app that you can use to make a week’s shopping list and menu for a week’s cooking. I want to do so much more, and it will happen as I get to it. All of this takes a lot of time and really boring repetition. I want to write more, and hopefully I’ll be able to some day. As for now, I’m looking for a job.