Stop learning alone!

Learn faster and stay on-track by joining this free class with other self-learners.

Register for Learning Vim from the inside now.

Learning Vim from the inside

Open Ended Class

Creator: pbr

Status: Established

Join this class!

Lesson 3: Assignment 3

Diving deep and fast into the source code

So the first thing we want to do is to assimilate what the developers have already written up for us as guides to help us understand how things work.

Start with looking at src/README.txt:

vim README.txt

First, follow the recommended steps in there

  • use Vim to create TAGS.
  • read through the readme
  • run :help development and read through that
  • run :help buffers if you're a vi user wondering what just happened and how to get around multiple windows

Next, investigate how :make works

This is an arbitrary goal but that's OK; right now we need to get used to poking around the code and learning what it's doing and how it does what it does.

  • theres a list of files in README.txt; learn and use the two-character command sequence to open the file which implements the ":make" command

  • how does ":make" work?

  • what's the name of the C routine which implements it?

  • what are the side effects of ":make" on your current files/buffers?

  • Extra Credit: whats that 'eap' argument all about? What is it and what does it contain?

Homework Submissions

7 total

shobhitjain (Self-grade: Outstanding)
Submitted 1 week ago | Permalink | Time spent: 1 minute
dioltas (Self-grade: Pretty good)
Submitted 2 years ago | Permalink

The first few steps were ok.

:make tags didn't work, first because I wasn't in the src directory.

then once in src/ it wouldn't work because ctags wasn't installed.

so I installed it by doing sudo pacman -Sy ctags.

It worked ok then.

I read through the readme, and :help development. I read about half of the :help buffers.

Tried out a few ctrl-]'s and gf's too.


I opened quickfix.c by placing the cursor on it and typing gf. Had a look through it, couldn't make much sense of it though! Couldn't find the make function either...

So I read up on the :make command, and if I understand correctly it's for compiling from within vim. It compiles what you specify by consulting a make file and if there is an error it shows it and places the cursor on the line of code that caused the error.

I'm only half guessing here though tbh.

So then to find the C routine that implements it, in the README.txt it says

The ":" commands are listed in ex_cmds.h.

So I jumped to ex_cmds.h using gf. I did /make to find make. Then I used ctrl-] to jump to ex_make.

I haven't been able to make much sense of ex_make(eap) yet. The function is used for ":make", ":lmake", ":grep", ":lgrep", ":grepadd", and ":lgrepadd" according to the comment.

And at the start there is a switch statement to figure out which is being used.

I think when :make is run it saves the current buffers to file. If there is an error then it should put the cursor on the line of the error I think. Not sure what other effects it has.

I did a few different lid -Sn eap commands to try and figure out what eap was about. Couldn't really figure it out. I'd guess that it's the location of the command / filename in memory as it's the only argument passed to a lot of the functions.

Comments:

dioltas
2 years ago

After reading through andrewferk's homework, I tried :tag exarg and found the definition in ex_cmds.h.

So it's a structure with info about the command such as arguments and flags and the next commands if there is one etc.

pbr
2 years ago

The main side-effect I was looking for people to be aware of was the saving-all-the-current-buffers that happens. I haven't seen the error-parsing-and-cursor-moving stuff in action (might intentionally break something, just to see it work) but that's a good catch - +1000 for finding yet another side-effect!

Regarding eap - anyone who's conversant with assembler and machine code knows EAP to be something very different than "Ex Arg Pointer" which is what I'm guessing is what it stands for in this case.

While most of the Vim source is pretty readable, I find abbreviations like this one to be obfuscating - and in this case, since there's already an assembler-level symbol called EAP it's even misleading. For example, you wouldn't name a variable that's a pointer to a storage table object "stop" because that's clearly something that should be a boolean indicator of whether to go or stop. I.e. it already HAS a symantic meaning so it shouldn't be used to mean something else.

BUT - for the most part - the Vim source code is pretty readable as far as C code goes.

Good diagnosis, and good usage of other people's homework to further your understanding. Looking forward to your contributions in the remaining lessons! -pbr

Sign up or log in to comment

sedrik (Self-grade: Could be better)
Submitted 2 years ago | Permalink

Starting from the src/README.txt under the Ex command section we can start looking for do_cmdline() or since we know :make is only one command jump stright to do_one_cmd() as mentioned in that section. do_one_cmd will eventually after some parsing and possible recursive calls call ex_make (:make has been replaced by :makeprg by this time).

The buffer will be flushed by the autowrite_all function.

eap is a struct defined in ex_cmds.h:1130 (or around that line depending on version of vim) that hold the arguments for the Ex command.

Now I don't know if it is just me that have not read C in a while but it is quite hard to follow. Thankfully by a mixture of tag jumping, documentation reading and cheating (read other peoples solutions) I came to this conclusion. :(

Comments:

pbr
2 years ago

Extra points for the tag jumping and documentation reading.

Regarding cheating... this form of learning is built into CrashCourse so I've got to guess it is an intentional move on Kevins part. People can get the clues they need to succeed if they have trouble with the assignment themselves. I guess that's part of what the self-grading is all about; you can get the right answer and grade yourself down if you peeked.

Sign up or log in to comment

webframp (Self-grade: Outstanding)
Submitted 2 years ago | Permalink
  • use vim to create TAGS: done
  • read through the readme: done
  • :help development: read it. #1 point - Read documentation first! Made mental note of vim special functions. Will probably have to check that list again. Assumptions section good to note.
  • :help buffers: read it. vim -o is great, vim -O is even better for my widescreen monitor.

  • :make - gf

  • :make uses the quickfix window, reading error messages from the errorfile or buffer. Inspired by the Manx Aztec C compiler for Amiga, error messages are saved to a file/buffer and jumped to one by one.
  • ex_make()
  • depending on "autowrite" option, changes in current buffer may be written out. If an error file with same name exists it will be deleted. The output of 'makeprg' is saved in the errorfile, which is then read thanks to the setting 'errorformat'
  • eap handles mapping of ex functions. the enum CMD_index defined in ex_cmds.h details the available names codes.

probably a few errors in that list, comments much welcome!

Comments:

pbr
2 years ago

Awesome. No errors that I noticed.

Best explanation so far for 'what does :make do'.

Best explanation so far for 'whats the eap'.

+10,000 for noting that the errorfile gets overwritten

Sign up or log in to comment

andrewferk (Self-grade: Outstanding)
Submitted 2 years ago | Permalink
  • I found this site very helpful: http://vim.wikia.com/wiki/Browsing_programs_with_tags
  • how does ":make" work? - ":make ***" eventually executes ex_make() found in 'quickfix.c'. This function executes make *** on the shell.
  • what's the name of the C routine which implements it? - the path to ex_make() is the following: main_loop() -> normal_cmd() -> do_cmdline() -> do_one_cmd() -> ex_make()... so it is implemented by do_one_cmd() in 'ex_docmd.c'
  • what are the side effects of ":make" on your current files/buffers? - Line 3937 in of ex_make() in quickfix.c calls autowrite_all(). Using Ctrl+] I was able to jump to the function and it is noted to "flush all buffers, except the ones that are readonly". So, the side effects would be that all buffers are flushed (except those that are readonly :).
  • Extra Credit: whats that 'eap' argument all about? What is it and what does it contain? - eap is an exarg struct which is used to pass around Ex commands. By using :tag exarg, you can quickly look it up.

Comments:

pbr
2 years ago

Thanks a lot for the reference to http://vim.wikia.com/wiki/Browsing_programs_with_tags - I'll read it and include it in this course (somewhere) based on what I see there.

You're on the right track with answers 1 and 2. However lets be clear; ex_make() implements the make functionality - do_one_cmd() calls the functionality.

Thanks for providing the call stack - did you trace it by hand, or run with a breakpoint, or?

thanks for sharing Ctrl-] with the class. Yes that jumps to the implementation of a function from anywhere it's being called. (So long as you've run :make tags)

So what happens when autowrite_all() encounters a buffer that's been modified and hasn't been written to disk? How's this affected by the 'autowrite' variable (as in ':set aw' and ':se noaw')

Good job identifying ':tag exarg' as a quick way to find the data structure. Note also you can see all the places exarg structs are used by running 'lid exarg_T' from the command line.

Sign up or log in to comment

mbudde (Self-grade: Pretty good)
Submitted 2 years ago | Permalink

If you get the following error when trying to make tags:
rm -f tags
*.c *.cpp if_perl.xs *.h
/bin/sh: arabic.c: not found
make: *** [tags] Error 127
then it is because you haven't run ./configure

  • To open the file move the cursor to the filename and press gf (goto file).

  • ex_docmd.c:do_one_cmd parses command line and figures out which command to run.

  • ex_docmd.c:replace_makeprg inserts makeprg in command line
  • quickfix.c:ex_make runs make command with do_shell

  • ex_make flushes all buffers with autowrite_all. I'm not quite sure what that function does though.

  • eap is a pointer to a exarg struct which is defined in ex_cmds.h. It contains stuff like the command and its arguments, etc.

Comments:

pbr
2 years ago

This is outstanding work; concise and correct. Great job, mbudde!

Thanks for the feedback on ':make tags' - it ought to succeed even if configure hasn't been run; you might consider opening a bug against Vim regarding this.

You've done an excellent job of describing what happens behind the scenes to implement ':make'. We'll want to dive into the 'eap' further though. One good way to understand it (IMO) is as a 'context'.

autowrite_all is the side-effect I was asking about. You identified the call itself; all that's left is to understand what it's doing and why. Here's my understanding: it tries to save all the buffers so that any changes you've made to various files are written to disk before the make gets called.

This saves you an immensity of time if you know it's there and are working on many files at the same time in multiple buffers. You don't have to switch to each one and save the file, then compile; simply run the ':make' and it'll save all the buffers first before running.

mbudde
2 years ago

Thank you :) I just read your reply to andrewferk and found out there's an 'autowrite' option. I tried running :make with some unsaved changes and it didn't work, so I got a little confused, but there is of course an option for that ;)

Sign up or log in to comment

cfedde (Self-grade: Outstanding)
Submitted 2 years ago | Permalink

Running a self compiled vim seems a bit more complex than just invoking it. Maybe compilation control with a $HOME prefix to ensure that all the support files are in the right places.

make distclean
./configure -prefix=$HOME
make -j4
make install

Comments:

pbr
2 years ago

Good point. I'm updating the 'build' assignment to reflect your note.

Sign up or log in to comment