The Art, Science, and Engineering of Programming

I’ve been relatively low key about something that I’m very excited about, so, after several months of planning, and a first, somewhat quiet, submission deadline, it’s time to release my enthusiasm. A group of us have started a new conference called <Programming>  associated with a new open-access journal called The Art, Science and Engineering of Programming. Here are some FAQs about this initiative.

Are there already agreements or plans for indexing the journal?

We are doing everything “by the book,” i.e. we have acquired an ISSN and will give DOIs to papers. We will make the metadata available for indexing services to ingest. It is our intention to have the journal indexed as widely as possible. Like other publishers, however, we cannot control which of those services choose to take our data in, or when. It usually takes a couple of years of sustained publication before some of these services start indexing new journals.

Why are you publishing the journal yourselves, as opposed to using an established publisher (e.g. Springer, Elsevier, Wiley)? Will my University recognize you as a publisher?

When we first started discussing the possibility of the journal, our first reaction was to send a proposal to a couple of existing publishers. Our proposal was accepted. But the longer some of us thought about it, the less appealing that arrangement felt. Mainstream academic publishers make money out of papers, so it was going to be very hard to offer affordable gold open access for our journal. Working with them would be a really lame move on our part.

It turns out that it’s not insurmountable to become a publisher. Perhaps the major hurdle is to create a non-profit organization that can act as a legal and financial entity, and then make sure that such organization files tax returns every year. But AOSA already existed, so that hurdle was gone. After that, there are a few things that need to be purchased for creating serial publications with DOIs, either once or every year, but it doesn’t seem to be a big deal. Granted, it’s more costly and more work than not having to do this at all! — but it’s not rocket science. So now AOSA is a non-profit publisher.

AOSA is not the first organization to become an independent journal publisher. I know of four such publishers in CS: JOT, VLDB, LMCS, and Microtome (JMLR). There are probably more. People seem to have accepted their existence. I admire these indie publishers: they are the ones showing what scholarly publications ought to be about.

I see that you accept different types of papers, and that I need to choose one type for my paper. What exactly are “The Art” papers? How are they different from engineering or scientific papers?

This begs a careful explanation, as not everyone agrees on the use of the words “Art,” “Science,” and “Engineering.”  Let me explain how we use them in this journal.

• Engineering perspectives are expected to contain concrete results of design optimizations, comparing improvements to a baseline.
• Theoretical science, or mathematics, is knowledge and technical skills acquired through mathematical formalisms. Examples include formal programming models and proofs.
• Empirical science is knowledge and technical skills acquired through controlled experiments and systematic observations. Examples include user studies and programming-related data mining.
• “The Art” is about our art of programming in the sense that Knuth uses that expression, which is a fairly technical activity (in Knuth’s case, using low-level machine language!). So: programming models, pearls, styles, languages, frameworks, libraries, APIs, architectures, as well as essays about our art.

Here is a rule of thumb: if you don’t have concrete measurable data or formal proofs backing up your design goals, then your paper is best tagged as “The Art.” These papers contain design principles that the authors believe bring benefits in some way, but beliefs are not facts: they’re, at best, hypotheses. Conversely, if your paper has claims of benefit, then it must back them up somehow, and that’s likely not going to be a “The Art” paper — more likely, it will be Engineering or empirical science. If the main contributions of your paper are formal proofs, then it’s theoretical.

Hypotheses and reflections are a critical part of the research process. I am going to make sure that reviewers understand that we don’t always need to have measurable data or proofs in order to make interesting contributions to programming. A lot of interesting design work is essentially a hypothesis that can only be validated by adoption in uncontrolled settings. Hence “The Art” category; it’s a safe space for properly done design work, as well as for essays about programming and our field.

Independent of their type, all papers have two things in common: they need to contribute something interesting and they need to be written in scholarly form. Scholarly papers must take considerable care in contextualizing the work. This may seem irrelevant, and an overhead, but it’s absolutely critical both for understanding the contributions of the work right now, and for understanding the work 30-50 years from now.

The paper classification is used for review, but it is not included in the final papers.

Is the <Programming> conference, and this new journal, just a new name for the AOSD/Modularity conference?

There is an historical and institutional link between one thing and the other, but the Programming journal/conference is a new initiative, not a continuation of the Modularity conference. The institutional link is that they are both sponsored by AOSA, a non-profit created in the early 2000s with the goal of promoting the interchange of ideas related to software development.

People involved with AOSA approached me last year to chair their new <Programming> conference, which they wanted to be different from Modularity. They were open to all sorts of new ideas, including starting a journal, so we realized it was a great opportunity to do, in one single scoop, the things some of us think have been missing from our research community, namely: (1) a journal-first-conference-later model of publication focusing on programming; (2) open access; and (3) a safe space for publication of design work and reflections. This goes way beyond what AOSD/Modularity was.

Is the <Programming> conference sponsored by the ACM or the IEEE?

The conference is sponsored by AOSA, in cooperation with ACM SIGPLAN and SIGSOFT.

Is presentation at the conference a requirement for publication in the journal?

For the time being yes, it’s required. In this aspect, we are following what VLDB does (“At least one author of every submitted paper is expected to attend the conference if the paper is accepted.”).

There are two reasons for making it a requirement: (1) a conference is a great way to have a sense of community, and to publicize people’s work; and (2) there are costs associated with running a journal; without a benefactor, the money needs to come from somewhere; rather than asking authors to pay a fee, we are including it as a line item in the conference’s budget, which is what CS conferences have been doing with the proceedings for the past 40 years.

Don’t get me wrong — I don’t like to travel, and I wish I would travel a lot less. Traveling to conferences is expensive and takes a lot of energy. Jet lag is a pain. Maybe soon we can replace physical conferences with virtual ones, so to get the best of both worlds — the community and the comfort of not traveling. (I actually proposed this to AOSA, but that’s where they drew the line! I haven’t given up on the idea yet, though)

I see there are several submission deadlines throughout the year. Are submissions open all the time, or just 2 weeks before each deadline?

Submissions are always open, you can submit any time. However, the reviewing process is batched on a per issue basis: we queue submissions for a given issue, establish a cut-off date, and once that date passes, we proceed to reviewing the papers in the queue. Submissions that enter the system past the deadlines are automatically queued to be processed in the next reviewing cycle (authors can continue to update those papers until that next deadline comes around).

Batching the reviewing process makes the process predictable for authors and reviewers. Everyone knows when to expect what.

Why only 3 issues per year? Why not more?

The main limiting factor here is reviewers’ time. We may add more issues in the future, with additional reviewing committees, chaired by additional associate editors. That’s in the radar of possibilities, but we need to start on a smaller scale before we increase the frequency.

How are the papers archived? How does arXiv come into the picture? 

We use arXiv to archive the papers for eternity — or at least while arXiv exists. Journals that do this are called “arXiv overlays” — there are several flavors. Here’s how it works for us. When we accept papers for publication, we ask the authors to upload their papers to arXiv, if they haven’t done it already. Then we ask the authors to share the paper password with us. We then get the paper sources from arXiv, add some information (e.g. DOIs), and upload a new version of the paper. That’s when the paper officially becomes part of the journal. We also link the Table of Contents from the journal’s web site to the arXiv page for the paper.

Additionally to arXiv, the Institute for Software Research here at UC Irvine is going to mirror the papers. Other institutions will do the same.