To reproduce, try:
DT <- data.table(x = as.complex(1:5))
DT[1]
## Error in `[.data.table`(DT, 1) : Unknown column type 'complex'
Similarly:
DT <- data.table(x = as.raw(1:5))
DT[1]
## Error in `[.data.table`(DT, 1) : Unknown column type 'raw'
The problem appears to be originating from the line:
ans[[target]] = .Call(CsubsetVector,x[[source]],irows)
Loosely related issue: #602