Makefiles are terrible tech. The problem is that they're slightly less bad than most other build system we've come up with, which makes them "useful" in a masochistic way.
Build systems tend to commit one or more of the following sins:
* Too basic: Once you try to build anything beyond a toy, it quickly becomes chaos.
* Too complicated: The upfront required knowledge, bureaucracy, synchronization and boilerplate is ridiculous. The build system itself takes an order of magnitude more data and memory than the build target.
* No standard library (or a substandard one that does things poorly or not at all): You must define everything yourself, leading to 10000 different incompatible implementations of the same build patterns. So now no one can just dive in and know what they're doing.
* Too constricting: The interface wasn't built as a simple layer upon an expert layer. So now as soon as your needs evolve, you have to migrate away.
* Too much magic: The hallmark of a poorly designed system. It doesn't have to be turtles all the way down, but it should be relatively close with few exceptions.
My 2c: Makefiles are excellent tech, just that a lot of people haven't learned to use it properly and use it as it was intended. I'm sure I'll get pushback, that's ok.
- Too basic: At least half of the software I use just uses plain makefiles and maybe a configure script. No autotools. I optionally run ./configure, and then make and make install, and it just works. I definitely wouldn't consider my setup to be a toy by any stretch of the imagination. It's built out of smaller programs that do one thing and one thing well.
- Too complicated: I don't know, I think make and how it works is really easy to understand to me at least. I guess everyone's had different experiences. Not necessarily your case, but I think usually it's because they had bad experiences that they probably blamed make for, when they were trying to build some complex project that either had a bad build setup itself (not make's fault), or without the requisite knowledge.
- No standard library: It's supposed to be tooling agnostic, which is what makes it universally applicable for a very wide range of tools, languages, and use cases. It's viewed as a feature, not a bug.
- Too constricting: I'm not sure what you mean here, it's designed to do one thing and one thing well. The simple layer is the dependency tracking.
- Too much magic: Cryptic or inconsistent syntax: See 'Too complicated'
The worst build systems are the ones centered on a particular programming language. Since there's N>>1 programming languages that's N>>1 build systems -- this does not scale, as the cognitive load is prohibitive.
The only general-purpose build system that spans all these languages is `make` or systems that target `make` (e.g., CMake). And this sucks because `make` sucks. And `make` sucks because:
- it's really difficult to use right
(think recursive vs. non-recursive make)
- so many incompatible variations:
- Unix/POSIX make
- BSD make
- GNU make
- `nmake` (Windows)
- it's rather ugly
But `make` used right is quite good. We're really lucky to have `make` for the lowest common denominator.
Gradle and Bazel are absolutely general purpose build tools that are very widely used in industry.
As a smaller contender, my personal favorite is the Mill build tool (written in Scala), that is basically what a build tool should be, it’s as close to a theoretical perfect as possible. I really advise reading the blog post by its author Li Haoyi: https://www.lihaoyi.com/post/SoWhatsSoSpecialAboutTheMillSca...
I've never seen anyone use Nix to actually build software; it's a glorified launcher for shell scripts in a sandbox, and typically is used to start the actual build system, such as make/cargo/go build/npm/etc, with known inputs.
Have you taken a look at using Nix as a build system? One thing I don't like about most build systems is the lack of a dependency check, C is most guilty of being the troublemaker here. But anyways, with Nix you can lock in dependencies and handle arbitrary feature flags and platforms as well.
Though it's possible this goes beyond your "just do stuff"
Xmake https://xmake.io/ for C and C++ (I haven't use that for anything serious yet) and Buck 2 https://buck2.build/ if you need a really complex build system. Both of these do caching of build artifacts and can do distributed builds (with less and more complex setup).
Not OP, but its not just that C/C++ lacks modules. I think that is missing the real issue. Any complicated program probably needs a custom developed tool to build it. As a simple example, imagine a program that uses a database - you want to keep the sources as SQL and generate classes from them. Thats a custom build step.
Its just that in some languages and build systems (Node, Maven), we have abstracted this away by calling them plugins and they probably come from the same group that made the library you need.
No such pluginsystem exists, as far as I am aware, for makefiles.
There are projects that generate files, depend on multiple languages, etc. If you push the job of a build tool to the compiler infrastructure, then why even have a “build tool” in the first place? Make is simply anemic for anything remotely complex, and there are countless better tools that actually solve the problem.
Yeah my biggest problem with make is that the compiler has to generate the header file dependencies. This means starting a C or C++ project with make from scratch is a hard problem and there is no default solution or default file for this other than to just use CMake.
Good luck writing Makefiles for Fortran, OCaml or (whenever they will really, actually work) C++ modules.
There aren't many widely used build systems that can handle such dynamic dependencies without some special "magic" for these, the only one that I know of (with a significant number of users, so not Shake) is Buck 2 (Bazel and all C++ build systems use "special magic", you can't write in user rules).
I've written Makefile for FORTRAN. Dealing with modules added about 6 extra lines. That was one of the more complex rules. Does that count as "special magic"?
If you've got your rule working for arbitrary named (i.e. not the name of the file) modules and submodules and an arbitrary number of modules and submodules generated by a single source file which uses FPP (whatever program that actually is ;) or CPP as preprocessor, then yes. And with "working" I mean adding each module file as a single target which is able to trigger a rebuild of the module.
You should be able to get that to work easier with GNU Make 4.3 and later, as that now supports grouped targets - which I have learned elsewhere in this forum. Now the only problém is getting your module dependencies without first compiling all files to generate the modules, as `gfortran -M` (and any other compiler that generates dependency information) AFAIK still doesn't "know" which file produces which module without actually generating the module files.
They all have the samé problem: that you don't know the name (or even the number) of modules (module files) being generated without reading the source.
And as a bonus every compiler uses a sligthly different naming scheme for the generated module file (this is of course no problem for OCaml ;).
As an example (using Fortran). File `test.f90`:
module first
contains
subroutine hello ()
end subroutine hello
end module first
module second
contains
subroutine world ()
end subroutine world
end module second
`gfortran -c test.f90` yields the following files (2 of them are modules):
-rw-r--r-- 1 roland staff 221 Sep 21 19:07 first.mod
-rw-r--r-- 1 roland staff 225 Sep 21 19:07 second.mod
-rw-r--r-- 1 roland staff 185 Sep 21 19:07 test.f90
-rw-r--r-- 1 roland staff 672 Sep 21 19:08 test.o
> Good luck writing Makefiles for Fortran, OCaml or (whenever they will really, actually work) C++ modules.
I've successfully written Makefiles for Fortran and they worked with ifort/ifx and gfort. In my experiments I've also made GNU Cobol, GNU Modula-2 and Vishap Oberon fit within the Makefile paradigm without much trouble. You have failed to provide reasons as to why those languages in particular (or more likely any language that's not of C heritage) can't be used with Makefiles. For instance, you can definitely couple OCaml with Makefiles, just use ocamlopt and treat .cmx files as object files, generated beforehand by ocamlopt -c (like you'd do with GCC). I am not familiar with C++ modules and as such I didn't experiment with them.
> I've successfully written Makefiles for Fortran and they worked with ifort/ifx and gfort.
Did the samé (I'm not sure if gfortran did exist at all at the time, I guess it had been g95), plus they worked with Absoft, PGI and Pathscale too (yes, that has been some time ago). And it was a great PITA. Not the least because at the time no Fortran compiler did generate the dependency description, so you either had to parse the Fortran sources by yourself or use makedepf90, which didn't work with all sources.
> You have failed to provide reasons as to why those languages in particular [...] can't be used with Makefiles.
I have obviously badly worded that. I didn't mean it is impossible, just that is a great PITA.
> I am not familiar with C++ modules and as such I didn't experiment with them.
They have the same problem, you don't know the name of the module that is going to be produced.
One or more, OK that leaves of course lots of room. I would estimate:
(too basic) Makefiles are not. (too complicated) They can be, depends on what you make them to be. (standard library) Well, there is one, there are some builtin functions you can use in the makefile. (too constricting) Haven't noticed that, so I would say no. (too much magic) Hmmm I don't see it. It is very clear what is a target and a dependency and so on. Not so magical. (syntax) Yeah definitely could be better. Even a plain JSON file would be better here.
How to compile and run this? We need a build system! Download and install GNU Make.
When that step is complete:
Type in
make hello
and its done. Now, run via ./hello
See, Too much magic (didn't even have a makefile or Makefile), no standard library, Too constricting, cryptic, too basic. And, because you had to install Make, too complicated. Hits every one of your objections.
Build systems tend to commit one or more of the following sins:
* Too basic: Once you try to build anything beyond a toy, it quickly becomes chaos.
* Too complicated: The upfront required knowledge, bureaucracy, synchronization and boilerplate is ridiculous. The build system itself takes an order of magnitude more data and memory than the build target.
* No standard library (or a substandard one that does things poorly or not at all): You must define everything yourself, leading to 10000 different incompatible implementations of the same build patterns. So now no one can just dive in and know what they're doing.
* Too constricting: The interface wasn't built as a simple layer upon an expert layer. So now as soon as your needs evolve, you have to migrate away.
* Too much magic: The hallmark of a poorly designed system. It doesn't have to be turtles all the way down, but it should be relatively close with few exceptions.
* Cryptic or inconsistent syntax.