FOR COMMENT ONLY Skel backends

[Mightyjo]

I have a backend code emission system working and ready for comment. I'll need to write a filter to replace M4 in our stack to use it directly the in skeletons, but I have an idea for how to build that from our buffer library.

I need eyes on this for now to help me see where I've gone blind from staring at it too long. What do your linters say?

Issues that came up along the way and I'll split out for the real PR:

• tests/array_nr was actually testing a reentrant case, so I renamed it array_r (important because of how the log_compilers use the suffixes)
• parallel make was breaking above -j16 and I traced it back to the removal of BUILT_SOURCES. I'll explain more fully in a separate PR.
• the Go backend was entirely fake so I turned it off for testing. It was just the C99 skeleton with C++ style comments.
  • I don't know Go well enough to write this skeleton myself. What other language do we want to support first?

Yes, structs of function pointers are hideous backdoor OOP. I used them as the least bad option I now. I'm open to learning better patterns.

Thanks all!

[westes]

I see that go has good standard and third party tooling for generating scanners, so please submit a pr to drop the go stuff as its own pr.

[westes]

tests/array_nr: Please submit this as its own pr.

[westes]

BUILT_SOURCES.: That'll teach me to go counter to what the automake manual recommends... I'm looking forward to that pr as well.

[westes]

replace M4 in our stack

We want that anyway, regardless of the work on this particular pr.

[Mightyjo]

BUILT_SOURCES.: That'll teach me to go counter to what the automake manual recommends... I'm looking forward to that pr as well.

The automake manual isn't very clear on what this variable does anyway. They should add an alias SYNCHRONIZE_ON_SOURCES for it to use in new code.

[Mightyjo]

Sorry the breakout PRs are taking so long. Really dug a hole for myself with the side projects.

I do want to eat some humble pie tonight. Looks like @Explorer09 and others have been right about how the Automake/Libtool folks mean for us to resolve build order and bootstrapping challenges. They've been updating their docs to make their opinion clearer and maybe hint at their plans. So I'm going to back off my position that BUILT_SOURCES is good or useful.

There's still a concurrency-related error message but I'll for an issue and help solve it another way. I'll add a note here when I do.

Happy New Year, everyone!

[Explorer09]

I do want to eat some humble pie tonight. Looks like @Explorer09 and others have been right about how the Automake/Libtool folks mean for us to resolve build order and bootstrapping challenges. They've been updating their docs to make their opinion clearer and maybe hint at their plans. So I'm going to back off my position that BUILT_SOURCES is good or useful.

I've lost the context of why you are using BUILT_SOURCES to fix the build problems.

I remember that almost a year go I've ditched BUILT_SOURCES in favor of explicit dependencies mentioned in the makefile.

The problem with BUILT_SOURCES was that, it was activated in every build target, even for partial builds like building libfl only, or when you intend to build only stage1flex and not other stages.

If you happen to use BUILT_SOURCES for "synchronization" then it still means something wrong with the build dependency tracking. I chose to ultimately remove the use of BUILT_SOURCES for this reason.

Happy New Year, everyone!

Happy new year, by the way.

[Explorer09]

Try adding this line in src/Makefile.am to fix the dependency problem:

cpp-backend.$(OBJEXT): cpp-flex.h

That way you won't need BUILT_SOURCES for the job.

[Mightyjo]

That's what I thought too, but make started telling me it was ignoring these rules with the message, "ignoring prerequisites on suffix rule definition." It is still running our instructions again, it's just not enforcing the strict ordering we specify. While I always see that message from make, the ordering doesn't seem to cause a problem for make -j 8 or less.

The most recent version of the Automake manual I read goes into more detail on this pattern than I've seen before. I expect between that and what you've already written we'll figure it out.

I'm going to drop that patch from this and the breakout I'm doing now.

[Explorer09]

This suggested change is nothing to do with the suffix rule.

The "suffix rule" in makefiles will look like this:

.c.o:
        $(CC) -c $(CFLAGS) $(CPPFLAGS) -o $@ $<

Note the two file extensions (.c.o) specified together.

What I suggested (cpp-backend.$(OBJEXT): cpp-flex.h) is to tell make about a dependency. Since cpp-flex.h is built, make needs to be told that cpp-backend.o cannot be made in parallel with cpp-flex.h, but the former needs to wait for the latter.

[Mightyjo]

You're right, those warnings are coming from tests/Makefile.am at lines 325, 328, and 362. Suffix rules I added a decade ago.

Working on a fix. Maybe they're what's been jamming my wide builds, idk.

[Explorer09]

https://github.com/westes/flex/pull/710/changes#diff-32103f666ff2fb42b025a47ccf1b959bbcc6db89f217e5943b1de73c81a4f9dbL325

A workaround is to use this:

TEST_SOURCES_L_C = \
alloc_extra_r.c \
alloc_extra_nr.c \
# more files to this list that depend on an .l source

$(TEST_SOURCES_L_C): $(FLEX)

.l.c:
	$(AM_V_LEX)$(FLEX) $(TESTOPTS) -o $@ $<

[Mightyjo]

tests/array_nr: Please submit this as its own pr.

See #716

[Mightyjo]

I see that go has good standard and third party tooling for generating scanners, so please submit a pr to drop the go stuff as its own pr.

See #717. I we can keep Go in the long run. Having tests for it on master while we don't really have a Go generator seems premature.

[Mightyjo]

I'll rebase this when the independent PRs have been dealt with.

[westes]

This is ready for you to rebase.