questions on mRMRe

I am trying this code on a rather small data created by myself. The data consist of 10 covariates and 100 observations. The first three are correlated among themselves and the first 5 are associated with outcome. The outcome is a survival object created by Surv. The survival object is in the first place in the data.
I am trying to use it and I have the following questions.

1. I am mostly interested in mRMR.ensample with method=’bootstrap’. My understanding is that solution_count parameter will implicitly set the number of boostrap resampling done. Since the survival object is on the first place in the data the target_indices=1. At this point I am only asking for feature_count=3. The question is why does this code does not work for a solution count of 125. It works for 100 but not for 125.

b=mRMR.ensemble(data = mydd, target_indices = c(1),method='bootstrap',solution_count=100,feature_count = 3)
b=mRMR.ensemble(data = mydd, target_indices = c(1),method='bootstrap',solution_count=125,feature_count = 3)
Error in .local(.Object, ...) :
user cannot request for more solutions than is possible given the data set

2. It seems that once it chooses one solution it gets stuck with it. I f I ask for 10 solutions and I run it twice I get:

b=mRMR.ensemble(data = mydd, target_indices = c(1),method='bootstrap',solution_count=10,feature_count = 3)
solutions(b)
$1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 3 3 3 3 3 3 3 3 3 3
[2,] 5 5 5 5 5 5 5 5 5 5
[3,] 6 7 7 7 7 7 7 7 7 7
b=mRMR.ensemble(data = mydd, target_indices = c(1),method='bootstrap',solution_count=10,feature_count = 3)
solutions(b)
$1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,] 3 2 2 2 2 2 2 2 2 2
[2,] 5 5 5 5 5 5 5 5 5 5
[3,] 6 9 9 9 9 9 9 9 9 9
But if I run it asking for 20 solutions I get:
b=mRMR.ensemble(data = mydd, target_indices = c(1),method='bootstrap',solution_count=20,feature_count = 3)
solutions(b)
$1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19] [,20]
[1,] 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
[2,] 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5
[3,] 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6
I would have expected to get a mixture of the two in all three instances.

3. Of a lesser importance is the following. When I run mRMR.classic and I am asking for more features than possible I obtain the outcome among the chosen features and it gets repeated until the number I asked for is filled. Here I have 10 covariates and I am asking for 13. Here it is what I get:

a=mRMR.classic(data = mydd, target_indices = c(1),feature_count = 13)
solutions(a)
$1
[,1]
[1,] 3
[2,] 5
[3,] 6
[4,] 2
[5,] 8
[6,] 7
[7,] 4
[8,] 11
[9,] 10
[10,] 9
[11,] 1
[12,] 1
[13,] 1
This is not a problem in itself but it becomes one when one runs simulations and the process becomes automatic. I am just wondering if this can be fixed.

thank you

13 February 2018
Thank you for looking into this issue. I have tried and it seems that selection differs now from run to run.

1. However the solution_count still has a limit on ow many i can ask:
   b=mRMR.ensemble(data = mydd, target_indices = c(1),solution_count=300,feature_count = 7)
   Error in .local(.Object, ...) :
   user cannot request for more solutions than is possible given the data set

2. Also when the solution_count is above a certain number the first solutions contain only 1 which in my case is the outcome.

b=mRMR.ensemble(data = mydd, target_indices = c(1),solution_count=30,feature_count = 7)
solutions(b)
$1
[,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22]
[1,] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 9 10
[2,] 1 1 1 1 1 1 1 1 1 1 2 3 4 5 6 7 8 9 10 11 3 3
[3,] 1 1 1 1 1 1 1 1 1 1 5 6 5 3 2 3 3 3 3 3 5 5
[4,] 1 1 1 1 1 1 1 1 1 1 8 8 6 6 5 5 5 5 5 5 2 2
[5,] 1 1 1 1 1 1 1 1 1 1 6 2 2 4 3 2 2 2 2 2 6 6
[6,] 1 1 1 1 1 1 1 1 1 1 7 11 8 8 9 6 6 8 6 6 8 8
[7,] 1 1 1 1 1 1 1 1 1 1 4 10 10 2 10 10 7 6 8 8 4 11
[,23] [,24] [,25] [,26] [,27] [,28] [,29] [,30]
[1,] 8 11 7 6 5 4 2 3
[2,] 3 3 3 2 6 5 5 5
[3,] 5 5 5 11 8 6 6 6
[4,] 2 2 2 3 3 3 8 2
[5,] 7 7 6 5 7 8 4 8
[6,] 10 6 10 8 10 7 3 7
[7,] 4 8 4 4 2 10 10 4

Thank you again. Sorry for being so insistent but I think this function has a lot of potential to help me.
Best regards.
Melania

[ba3lwi]

Thank you for pointing out to these issues.
We have fixed the first two issues and now you could re-install the package from github as follows to get the latest updates:

library(devtools)
install_github("bhaibeka/mRMRe", ref="master")

The first issue was a check that is specific to only method=’exhaustive’ but we had it called for both methods.

The second issue is actually not an "issue". We seed the randomization of bootstraps using the number of seconds till current date. However, since the method is very quick and your data set is not that big, all of the bootstraps happened in the same second and hence got the same seed for the random sampling and that is why you got the same solutions. We have updated the code to use the number of microseconds till current date as the seed for each bootstrap sampling.

We will update you once we resolve the 3rd issue. Thank you for reaching out with these issues and please let us know if you experience any other problems.

[kingcrimsontianyu]

Thank you for pointing out to these issues.
We have fixed the first two issues and now you could re-install the package from github as follows to get the latest updates:

library(devtools)
install_github("bhaibeka/mRMRe", ref="master")

The first issue was a check that is specific to only method=’exhaustive’ but we had it called for both methods.

The second issue is actually not an "issue". We seed the randomization of bootstraps using the number of seconds till current date. However, since the method is very quick and your data set is not that big, all of the bootstraps happened in the same second and hence got the same seed for the random sampling and that is why you got the same solutions. We have updated the code to use the number of microseconds till current date as the seed for each bootstrap sampling.

We will update you once we resolve the 3rd issue. Thank you for reaching out with these issues and please let us know if you experience any other problems.

Dear developers,

Using mRMRe v2.1.0, I still ran into the problem that OP reported on 13 February 2018, where the index of the outcome repeatedly and erroneously show up in the solution matrix.

I'm not sure if in R dev community there exists a canonical way to initialize random numbers for bootstrap sampling, but using time-dependent random number seeds would cause trouble to reproducibility. My two cents are the seeds should be set incrementally, i.e. a certain value S plus bootstrap sample index. S should default to a time-dependent random integer, but the API should also allow users to set the value of S.