Enabling OpenCL causes access violation on a certain net structure #85
Description
First ... Great Work! I'm still struggling a bit since that is a BIG library but I think it's great!
Following your examples I tried to use the lib on a ECG problem using the following code:
NN := TNNet.Create;
inputLayer := TNNetInput.Create(numFeatures);
NN.AddLayer( inputLayer );
NN.AddMovingNorm(False, inputLayer);
NN.AddLayer( TNNetConvolutionReLU.Create( 20, 50, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetConvolutionReLU.Create( 20, 50, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 30, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 30, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 24, 10, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
NN.AddLayer( TNNetConvolutionReLU.Create( 12, 10, 0, 1 ) );
NN.AddLayer( TNNetMaxPool.Create( 2 ));
NN.AddLayer( TNNetDropout.Create( 0.25 ) );
// this layer is by mike:
NN.AddLayer( TNNetFullConnectLinear.Create(2) );
NN.AddLayer( TNNetSoftMax.Create );
NeuralFit := TNeuralFit.Create;
NeuralFit.FileNameBase := 'ECG_' + FormatDateTime( 'ddmmyy_hhnn', now );
NeuralFit.InitialLearningRate := 0.01; // learning rate of 0.1 does not seem to work properly
NeuralFit.LearningRateDecay := 0.001;
NeuralFit.StaircaseEpochs := 30;
NeuralFit.Inertia := 0.9;
NeuralFit.L2Decay := 0.00001;
NeuralFit.MaxCropSize := 0;
NeuralFit.InferHitFn := ClassCompare;
NeuralFit.LossFn := TNeuralFitHack(NeuralFit).DefaultLossFn;
netSave := TNotifyEvtObj.Create(NeuralFit);
NeuralFit.OnAfterEpoch := netSave.OnAfterEpoch;
EasyOpenCL := TEasyOpenCL.Create();
if EasyOpenCL.GetPlatformCount() > 0 then
begin
WriteLn('Setting platform to: ', EasyOpenCL.PlatformNames[0]);
EasyOpenCL.SetCurrentPlatform(EasyOpenCL.PlatformIds[0]);
if EasyOpenCL.GetDeviceCount() > 0 then
begin
EasyOpenCL.SetCurrentDevice(EasyOpenCL.Devices[0]);
WriteLn('Setting device to: ', EasyOpenCL.DeviceNames[0]);
NeuralFit.EnableOpenCL(EasyOpenCL.PlatformIds[0], EasyOpenCL.Devices[0]);
end
else
begin
WriteLn('No OpenCL capable device has been found for platform ',EasyOpenCL.PlatformNames[0]);
WriteLn('Falling back to CPU.');
end;
end
else
begin
WriteLn('No OpenCL platform has been found. Falling back to CPU.');
end;
NeuralFit.Fit(NN, EDRTrainingVolumes, EDRValidationVolumes, EDRTestVolumes, {batchsize=}256, {epochs=}400);
NeuralFit.Free;
netSave.Free;
NN.SaveToFile('NN.dat');
Input is 10 seconds of ECG aka 1000samples.
It seems that it may happen that fHasOpenCL is true but fShouldOpenCL is false when the base function
TNNetConvolutionBase.EnableOpenCL is called.
This leads to an unassigned fDotCL object which then raises an exception.
I needed to change TNNetConvolutionBase a bit to get it started (though I don't know if it is correct):
{$IFDEF OpenCL}
procedure TNNetConvolutionBase.EnableOpenCL(DotProductKernel: TDotProductKernel);
begin
inherited EnableOpenCL(DotProductKernel);
// fDotCL is not assigned in case fShouldOpenCL is false
if Assigned(FDotCL)
then
FDotCL.PrepareForCompute(FConcatedWInter, FInputPrepared, FVectorSize)
else
FHasOpenCL := False;
end;
let me know if that fix is ok for you.