The technical merits of languages just aren't relevant to choosing them for most developers, unless they're helping solve a people problem.
"Artisanal" languages like Lisp, and Forth can be fantastic at solving problems elegantly, but that's not the most important thing to optimize for in big organizations where a large portion of your time is spent reading code written by people you've never met who may not have known what they were doing.
Many of the tools that come from big tech are designed to ease the challenges of organizational scale. Golang enforces uniform styles so that you don't have idiosyncratic teams doing their own things. Bazel is a largely language agnostic build system, with amazing build farm support. Apple and Google have both contributed heavily to sanitizers and standard library hardening in order to detect/eliminate issues without reading the code. Facebook has poured vast resources into automatic static analysis. AWS built an entire organization around treating all their internal interfaces the same as external ones.
I think the field of programming languages has grown enough that we have to start acknowledging the future of programming largely won't be in the context of what it means for devs working at large corporations. One of my favorite talks is from Amy J. Ko called A Human View of Programming [1], which argues there are many other ways to look at programming than "tool for generating business activity" and "mathematical construct", which heretofore have been the dominant views of programming languages.
Because there are so many other forms and purposes programming languages can and will take (she goes through them in the talk), so evaluating them and creating them solely on how well they are able to fit into a corporate R&D pipeline is a very narrow and short-term view of the field.
Indeed, it's been the case for a long time now that most people who write programs are not in fact professional software developers. The most used language in the world is Excel, by several orders of magnitude, and it's the opposite of everything devs say a "proper" language must be. There's something we as a field still need to learn from that.
Is the most used bridge-building technique a plank over a small culvert, or the properly engineered bridge that carries constant, multi-lane highway traffic for a century? How do we weigh the usage of resulting products into the usage of a design and production method? Should we consider the number of program users? The users X hours of usage?
Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods. In some sense, the domains and domain rules for material goods emerge out of the connection to culture, economic roles, health, and safety aspects. Whether it falls into civil engineering, building codes, transporation rules, consumer product safety, food and drug, ...
The self-similar way that software can be composed into systems also makes it confusing to categorize. Imagine if we talked about other crafts the same way, and conflated textile manufacturing, clothing design, tailoring, costume making, wardrobe management, scripting, choreography, acting, and dancing as a single field that coordinates the visual movement of fabric on a stage.
Define it as # of people who possess the knowledge and resources to effectively use said language to solve a problem they have in their actual lives.
> Fundamentally, the software field is still just so young and we haven't teased apart the "obvious" different domains and domain rules that we have for production of different material goods.
I think we're saying the same thing here from different angles. I said it's developed enough that we can see there are very different ways of doing things. You said it's young enough that we don't know all the different things there are.
Instead, the world should have seen the light of Hindley-Milner type systems, ML-inspired languages, immutability, or at least not sharing mutable state. Did Haskell fail? Hmm, let's look at Typescript and Rust.
Don't get me wrong, a Lisp is always a great and fun language, and you can write whatever DSL you might like on top of it. But the old joke that "a Lisp programmer knows the value of everything, and the cost of nothing" still has quite a bit of truth to it.
Being able to do something like:
(deftype Digit ()
'(Integer 0 9)
(deftype Digit-Vector (&optional (length '*))
`(Vector Digit ,length))
(defun integer->digit-vector (integer)
(declare (type Integer integer))
(coerce (loop :for character :across (format nil "~D" integer)
:for digit := (char-digit-p character)
:when digit
:collect digit)
'Digit-Vector))
Spec is definitely underrated here considering it's built into the language and has a wider scope but for most people they want the intellisense experience which you can get with clj-kondo + mailli but is not built in so most teams don't use it, fair enough
I'd like to move the goal posts though and say I want flowstorm in every (any other?!) language
I can just run the program and scrub backwards and forwards through the execution and look at all the immutable values frame by frame with a high level UI with plenty of search/autocomplete options
For program understanding there's nothing better
The fact I can program against the timeline of values of my program and create custom UI on top is crazy
One of the most mind blowing demos to me was Bret Victor's inventing on principle and having a programmable reverse debugger for your language makes those demos viable
I built an emulator recently for work that replays what happens on live locally, combined with flowstorm I can go line by line and tell you exactly what happened and why, no print statements no reruns with my own custom UI customised to our apps interesting parts
This is my appeal to anyone outside of Clojure please build flowstorm for JavaScript and or Python
The design of flowstorm is definitely helped by the fact that 95% of Clojure programs are immutable but I don't think it's impossible to replicate just very difficult
But the real strength of Lisp is in the macros, the metaprogramming system. And I suspect that typing most macros properly would be a bit less trivial than even typing of complex generic types, like lenses. Not typing a macro, and only typechecking the macroexpansion would formally work, but, usability-wise, could be on par with C++ template error reporting.
I think we (the Clojure community) quickly figured out we don't really want static typing, which is a bit evident by the low uptake of Typed Clojure.
Personally I found it to A) make it a hassle for downstream consumers since your design is suddenly impacting others, because you can "lock things down" and B) have that very same effect on your own codebase, where it becomes a lot less flexible where it needs to be flexible.
Nowadays, I just use another language if I want static types, which happens sometimes but not nearly as often to say that dynamically typed languages are "dead" or whatever.
To this day, I know of no study that was able to demonstrate superiority of statically-typed languages - [1].
What studies clearly show, is that both in terms of productivity [2] and bug reduction [3], expressivity reigns supreme.
And Clojure is the most expressive [4] out of languages that can leverage huge ecosystems (Java and JS, soon C++ through Jank dialect).
[1] https://danluu.com/empirical-pl/ [2] PBX study from Economics of Software Quality by Caper Jones [3] https://arxiv.org/pdf/1901.10220 [4] https://redmonk.com/dberkholz/2013/03/25/programming-languag...
Probably not an absolute truth, but definitely a personal truth for you. For me, it's pretty much the opposite, static/fixed types is such a pain when you just wanna solve a problem and you know how to achieve it, all the invariants/constraints but the language tells you "No, you know what, this other person said you cannot use X for Y, so I'm gonna say no" instead of just letting me do that thing.
With that said, I still reach for Rust for about ~30% of new projects, despite the types, because some languages fit other problems better, simple as that. And still a lot more contracting gigs available for various Rust codebases who've fallen into disrepair, so one does what one can.
I feel like big codebases regardless of their size are hard to wrangle not because of the languages used, but because of the programmers having to rush through building proper abstractions, or even considering not adding so much abstractions. I've seen awful heavily typed codebases as much as I've seen awful dynamically types codebases or awful codebases not using explicit types anywhere, to me there seems to be no correlation between "awful" and "number of explicit types used".
Personally, I prefer a big codebase with lots of (good) unit tests in a dynamic program, than a that same big codebase with no unit tests and explicit static typing everywhere, especially when refactoring and needing to ensure everything (from a business logic perspective) works correctly. But again, this is my personal truth, and I'm not trying to claim it's universal.
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world."
This is me to a T — even when I'm building hobby projects. The point of writing any code, for me, is most of all to see a certain idea to fruition, so I choose what will make me most productive getting where I want to go. And while I still worship at the altar of Common Lisp as an incredibly good language, the language matters much less than the libraries, ecosystem, and documentation for productivity (or even effective DSL style abstraction level!), so eventually I have had to make my peace with Python, TypeScript, and Rust.
An incredible epiphany that you can't transmit may not be as useful as a a moderately clever idea you can.
Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions, but they also increase cognitive load and ramp up cost. In most orgs with churn and constant hiring, you do not get to enjoy "we built great abstractions and now we are fast". You live in "someone new is trying to understand what the last person did".
That is why hand holding and guard rails win. Not because Python or similar are technically superior, but because they support a commoditised, fungible workforce. Even if a wizard in a high dimension language is 2x more productive, that does not necessarily beat a slightly larger team in a mainstream language once you factor in turnover and ramp up. Companies mostly optimise for business impact, predictable delivery, and ease of staffing, not for maximising the ceiling of the top few programmers.
That said, at the individual level, as a programmer, you definitely can benefit from learning and mastering those added dimensions, even if you are to never use them again professionally, they expand your mindset.
In a language like C++, Rust or even Haskell, those "powerful abstractions" are mostly about building libraries, not application code. You still benefit from powerful libraries that were built as part of the ecosystem, even while keeping your own application code simple and intuitive to ease onboarding of new coders. Part of the "powerful abstractions" is to enable a "hand-holding and guard rails" approach for application code that's going to interface with that full-featured library.
Using a bad language for 8 hours a day makes me irritable and it's impossible to prevent that irritability from overflowing into my interactions with other people. I'd rather that my conversations with the computer be joyful ones.
I'm going through this now, having to deal with code nobody wants to touch because it is overly complex, has no documentation, and is in a language no one else knows. Now, whenever i see an effort like this, to bring an exoteric language for absolutely no good reason, i try to kill it as fast as possible.
I don't want to be the victim of this code in the future or have my team bear the cost of maintaining stuff they don't understand.
Programming language ergonomics matter and there is a reason why Lisp has so little adoption even after a half a century.
I could probably include Forth as a scripting language in a bigger app, but that app is probably going to want more complex variables than machine word size ints, and fixed-length strings. So, oh dear, Forth's not a great fit for that, and everyone just uses Lua anyway, so Lua it is.
Which is a pity, because I like Forth, and I used to to create possibly the nerdiest project on Github. I like Forth a lot, and I'd encourage anyone curious about how you get from "chunk of thinking sand and copper" to "thing I can type commands in" to have a crack at it - it's easy enough to implement your own, just to see how it's done.
But I don't expect anyone else to jump up and like it too, just because I said it's cool.
Well, maybe one or two of you will?
It boots off a floppy disk, so it was really just a case of working out where in the ROM and OS disassembly the entry points were, and making it all fit around the assumptions the ROM makes.
This allowed me to make up a diagnostics disk to check the RAM (a whopping 128kB of sample RAM) and IO are working.
https://gordonjcp.github.io/miragetools/
Now I wouldn't place a bet on it being the nerdiest project on Github, but I suspect it's well up there.
i think lisp's magic is a lot more cultural than most people think. i.e. how lispnicks implement lisps and the ecosystem around it. how easy it is to walk the entire ladder of abstractions from machine code to project specific DSL's. how pluggable its parsing pipeline is -- something that is not even exposed in most languages, let alone customizable.
the language, the foundation, of course matters. but i think to a lesser extent than what people think. (hence the trend of trying to hire lispnicks to hard, but non-lisp positions?)
and it's not even an obviously good culture... (just how abrasive common lispers are? need to have a thick skin if you ask a stupid question... or that grumpy, pervasive spirit of the lone wolf...?)
maybe it's just a peculiar filter that gets together peculiar people who think and write code in peculiar ways.
maybe it's not the macros, but the patterns in personality traits of the people who end up at lisp?
The talk I posted from Alan Kay is the steel man. I think you've missed the essence of TFA because it's not really about Clojure or lisp.
I’m not just talking about people who program for a living. The majority of academic CS chooses its research directions because of what limits people are running into for business; even privacy-focused software has been commoditized by many business; a large amount of OSS development is by (and often paid for by the employers of) people working for money; heck, after Linus’s initial “just a hobby OS” period even Linux’s contribution base was primarily driven by business needs (even if those needs influenced it via “contributor had a problem at work and committed a solution for it upstream in spare time” as often as “paid contributor upstreamed a change on behalf of their employer”).
But look at historical languages and why they were created:
Algol - to explore writing algorithms
Fortran - to help scientists write programs using typical math formulas
Matlab - to help write programs in linear algebra
Haskell - to explore lazy program evaluation
ML - to explore how to reason about proof automatically
C - to build an OS
Python - to interface with an OS
LISP - to formalize symbol processing
APL - to explore programs defined over arrays
LOGO - to help young kids to program computers
Prolog - to create a language around the idea of formal logic.
Smalltalk - to create an entire programming system, not just a language
(I've left out C++, Java, and JavaScript because I feel like those languages are mostly about serving business interests)
Pretty much the entire computing landscape over the past 50-70 years has been defined by people writing languages for reasons other than "this is for a business to use to make more money". So if we let business-driven interest dictate the future direction, we will have missed out on all the things that could have been. Would Haskell ever have been invented if businesses interests were the only concern for researchers?
The interesting bit here isn't related to Clojure or Lisp, that's what people are chewing on because it's the surface level topic of the essay. The thing that interests me about this post is how it touches on the psychedelic nature of learning programming languages and what that does to one's perspective as a programmer.
So when you ask "what business thoughts can you express in the language", my response is it's not about what you can express, but more about "what new thoughts / ways of thinking has the experience of learning the language caused you to become aware of?".
Few people can go their whole lives writing Python and think all the possible thoughts there are to think about the shapes and forms programming can take. It's hard to develop a good sense for your own practice of programming if you never step outside and see it from other perspectives. It often takes exposure to completely new languages with different design points and abstractions to really give one perspective about their own practice.
The easiest one related to lisp is just the form of the syntax, which is surprising to many students. Most programmers don't even consider you can write (+ 1 1) and that's the same thing as (1 + 1). They don't think about the pros, or the cons, or why one might be better than the other, because every language they've used and ever will use writes it as (1 + 1). But as soon as they see Lisp, they immediately see something that changes their perspective about everything they have previous learned, and therefore will reshape how they approach programming in the future. It doesn't have to mean they will use Lisp going forward, but it does mean they will program in their language of choice with greater purpose. That's how we each hone our craft.
Add on homoiconicity, read/eval, programming as manipulating an AST, and meta programming, and you've got yourself a righteous trip.
Now, can people learn about those things from other sources, without encountering Lisp? Of course. Can you express those ideas in mainstream languages? Yes. But the point is many devs don't think those thoughts until they've had a psychedelic experience, it's very common, that we need to consider that it's part of one's growth journey as a programmer to have these experiences, so we should encourage it. The author of TFA doesn't have experience enough to make that point, as they are on the journey, which is why I brought Alan Kay's talk into context, since his perspective is from the other end of the journey.
If you only think about programming languages as a way to make money, the analogy of being stuck in Flatland is perfect.
But TFA was riffing on Paul Graham's old essay Beating the Averages, which argued precisely that the expressiveness of Lisp gave his startup a business edge. That was the context of my comment. I'd add that most of what most of us do in our day jobs is to use programming languages to make money, and there's no shame in that at all. And if you want to talk about why certain languages get widespread adoption and others not, you have to talk about the corporate context: there is no way around it.
But I'll rephrase my question, just for you: "what abstract problems can you solve or thoughts can you express in Clojure that you can’t express in Python or Rust?"
I think the money question is a red herring here. I’d phrase it more like: what problem in a user’s problem space is expressible only like this? And if the only user is the programmer, that’s alright, but feels more aligned with pure academia. That’s important, too! But has a much smaller audience than engineering at large.
You mean another slick text editor :/
It might surprise the author to learn that there are many people who:
1) Have tried lisp and clojure
2) Liked their elegance and expressiveness
3) Have read through SICP and done most of the exercises
4) Would still choose plain old boring easy-to-read always-second-best Python for 90% of use-cases (and probably Rust for the last 10%) when building a real business in the real world.
The article could really benefit from some steel-manning. Remove the cute Flatland metaphor and it is effectively arguing that lisp/clojure haven’t been universally adopted because most programmers haven’t Seen The Light in some sort of epiphany of parentheses and macros. The truth is more nuanced.