Implement joins using secondary keys

[arunsrinivasan]

• X[Y] gains an on = argument (as proposed by Matt).
• merge will not copy anymore once implemented.

Update posts:

• Update this SO post as well.
• And this

[jangorecki]

Maybe on can be further extended to handle left, right, full joins?
The possible extension which comes to my mind (kind of oracle old non-ansi join syntax):

X[Y, on=col] # default join - currently right outer, from 2.0 inner join(?)
X[Y, on=col+] # right join
X[Y, on=+col] # left join
X[Y, on=+col+] # full outer join

I'm sure we can figured out the better, this would finally closes #614

[clarkdk]

! or - for a non-join?

X[Y, on=!col]
X[Y, on=-col]

[eantonya]

Honestly, I think these suggested "on" decorators (i.e. the +,-,!) are not a good idea. As I see them right now, they violate the intuition of thinking of X[Y] as a kind of extended subset operation.

For your reference, I read X[Y] as "take the Y rows of X" and X[!Y] as "take the rows of X that are not Y". You can't keep that subset reading for e.g. a full join as far as I can tell.

Convince me that my intuition should change and provide a new reading for the suggested syntax.

[stephlocke]

on is useful in that it allows on the fly key setting for joins, reducing lines of code needed.

I see the benefit in that it extends @eantonya 's reading by saying "take the Y rows of X where we define Y as being unique based on columns/attributes A,B,C" (currently that last bit is implied not explicit in basic syntax).

Not sure I like the plus operator as a means of specifying behaviour - I'd prefer an explicit join parameter which could take left, right, etc

I'd also really love to see 1.9.6 on CRAN soon so I can use it in production ASAP - will adding this feature delay significantly?

[clarkdk]

For me, the point of this FR is 1) to be able to explicitly specify the columns used for the join, and 2) that the specified columns will automatically use/create secondary keys, if they are not the primary key.

To be fully general, the syntax should also 3) allow join columns to be given separately for X and Y, in case they don't have the same names in both tables [equivalent to by.x= and by.y= in base merge]. "by" has a different use in data.table (grouping), so there is a need for a different keyword. Hence, on=, and perhaps on.x= and on.y=, too.

Being able to specify left, right, full, outer, non-join etc. would be nice, but that is a separate FR (#614) which has not been given priority or milestone yet. I assume the "join" keyword is reserved for that FR.

[MichaelChirico]

Throwing my hat in the ring of support for this, would clean up this process of mine substantially.