[mpich-discuss] Running an mpi program that needs to access /dev/mem
Jim Dinan
james.dinan at gmail.com
Fri Jun 14 15:31:52 CDT 2013
I don't know if this has been suggested, but you could also add your user
to the kmem group and chmod /dev/mem so that you have the access you need.
~Jim.
On Fri, Jun 14, 2013 at 1:24 PM, Pavan Balaji <balaji at mcs.anl.gov> wrote:
>
> You can run mpich as root. There's no restriction on that. You still
> haven't tried out my suggestion of running "id" to check what user ID you
> are running your processes as. My guess is that you are not setting your
> user ID correctly.
>
> -- Pavan
>
>
> On 06/14/2013 06:27 AM, Lee, Eibhlin wrote:
>
>> I found that the reason we want to access /dev/mem is to setup memory
>> regions to access the peripherals. (We are trying to read the output of an
>> ADC). At this point it becomes more a linux/raspberry-pi specific problem
>> than an MPICH problem. Although the fact that you can't run a program that
>> needs access to memory mapping (even as the root user) seems something that
>> MPICH could improve on for future versions. I know I am using smpd instead
>> of hydra so this problem may already be solved. But if someone could
>> confirm that, it would be really helpful.
>> ______________________________**__________
>> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
>> Lee, Eibhlin [eibhlin.lee10 at imperial.ac.uk]
>> Sent: 14 June 2013 11:20
>> To: discuss at mpich.org
>> Subject: Re: [mpich-discuss] Running an mpi program that needs
>> to access /dev/mem
>>
>> Gus,
>> I tried running cpi, as is included in the installation of MPI, on two
>> machines with two processes. The output message confirmed that it had
>> started only 1 process instead of 2.
>> Process 0 of 1 is on raspi
>> pi is approximately...
>>
>> Then it just hung. I think this is because the other machine didn't know
>> where to output the data?
>>
>> When I tried running two processes on the one machine using the wrapper
>> you suggested the output was the same but doubled. It didn't hang. This
>> confirms that every process was started with rank 0.
>>
>> I'm not entirely sure why /dev/mem is needed. I'm working in a group and
>> another member set up io and gpio and it seemed it needed access to
>> /dev/mem I am going to do a strace as suggested by Pavan Balaji to see
>> where it is used and see if I can somehow work around it.
>>
>> Thank you for your help.
>> Eibhlin
>> ______________________________**__________
>> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
>> Gus Correa [gus at ldeo.columbia.edu]
>> Sent: 13 June 2013 21:11
>> To: Discuss Mpich
>> Subject: Re: [mpich-discuss] Running an mpi program that needs to
>> access /dev/mem
>>
>> Hi Eibhlin
>>
>> On 06/13/2013 12:59 PM, Lee, Eibhlin wrote:
>>
>>> Gus,
>>> I believe your first assumption is correct. Unfortunately it just seemed
>>> to hang. I think this might be because each one is being made to have the
>>> same rank...
>>>
>>
>> Darn! I was afraid that it might give only rank 0 to all MPI processes.
>> So, with the script wrapper the process being launched by mpiexec may
>> indeed be sudo,
>> not the actual mpi executable (main) :(
>> Then it may actually launch a bunch of separate rank 0 replicas of your
>> program,
>> instead of assigning to them different ranks.
>> However, without any output or error message, it is hard to tell.
>>
>> No output at all?
>> No error message, just hangs?
>> Have you tried a verbose flag (-v) to mpiexec?
>> (Not sure if it exists in MPICH mpiexec, you'd need to check.)
>>
>> Would you care to try it with another mpi program,
>> one that doesn't deal with /dev/mem (a risky business),
>> say cpi.c (in the examples directory), or an mpi version of Hello, world,
>> just to see if the mpiexec+sudo_script_wrapper works as expected or
>> if everybody gets rank 0?
>>
>>
>> It may already be obvious but this is the first time I am using Linux. I
>>> had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both
>>> without success.
>>>
>>
>> "which mpiexec" will return the path to mpiexec, but won't execute it.
>>
>> You could try this (with backquotes):
>>
>> `which mpiexec` -n 2 ~/main
>>
>> On a side note, make sure the mpiexec you're using matches the
>> mpicc/mpif90/MPI library from the MPICH that
>> you used to compile the program.
>> Often times computers have several flavors of MPI installed, and mixing
>> them just doesn't work.
>>
>> Is putting the full path to it similar to/is a symlink? (This still
>>> doesn't make main have super user privileges though.)
>>>
>>
>> No, nothing to do with sudo privileges.
>>
>> This suggestion was just to avoid messing up your /usr/bin,
>> which is a directory that despite the somewhat misleading name (/usr,
>> for historical reasons I think),
>> is supposed to hold system (Linux) programs (that users can use), but
>> not user-installed programs.
>> Normally things are that are installed in /usr get there via some Linux
>> package manager program
>> (yum, rpm, apt-get, etc), to keep consistency with libraries, etc.
>>
>> I belive MPICH would install by default in /usr/local/ (and put mpiexec
>> in /usr/local/bin),
>> which is kind of a default location for non-system applications.
>>
>> The full path suggestion would be something like:
>> /path/to/where/you/installed/**mpiexec -n 2 ~/main
>>
>> However, this won't solve the other problem w.r.t. sudo and /dev/mem.
>>
>> You must know what you are doing, but it made me wonder,
>> even if your program were sequential, why would you want to mess with
>> /dev/mem directly?
>> Just curious about it.
>>
>> Gus Correa
>>
>>
>>
>> Eibhlin
>>> ______________________________**__________
>>> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf
>>> of Gus Correa [gus at ldeo.columbia.edu]
>>> Sent: 13 June 2013 15:37
>>> To: Discuss Mpich
>>> Subject: Re: [mpich-discuss] Running an mpi program that needs to
>>> access /dev/mem
>>>
>>> Hi Lee
>>>
>>> How about replacing "~/main" in the mpiexec command line
>>> by one-liner script?
>>> Say, "sudo_main.sh", something like this:
>>>
>>> #! /bin/bash
>>> sudo ~/main
>>>
>>> After all, it is "main" that accesses /dev/mem,
>>> and needs "sudo" permissions, not mpiexec, right?
>>> [Or do the mpiexec-launched processes inherit
>>> the "sudo" stuff from mpiexec?]
>>>
>>> Not related, but, instead of putting mpiexec in /usr/bin,
>>> can't you just use the full path to it?
>>>
>>> I hope this helps,
>>> Gus Correa
>>>
>>> On 06/13/2013 10:09 AM, Lee, Eibhlin wrote:
>>>
>>>> Pavan,
>>>> I had a lot of trouble getting hydra to work without having to enter a
>>>> password/passphrase. I saw the option to pass a phrase in the mpich
>>>> installers guide. I eventually found that for that command you needed to
>>>> use the smpd process manager. That's the only reason I chose smpd over
>>>> hydra.
>>>> As to your other suggestion. I ran ./main and the same error (Can't
>>>> open /dev/mem...) appeared. sudo ./main works but of course without
>>>> multiple processes.
>>>> Eibhlin
>>>> ______________________________**__________
>>>> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf
>>>> of Pavan Balaji [balaji at mcs.anl.gov]
>>>> Sent: 13 June 2013 14:34
>>>> To: discuss at mpich.org
>>>> Subject: Re: [mpich-discuss] Running an mpi program that needs to
>>>> access /dev/mem
>>>>
>>>> I just saw your older email. Why are you using smpd instead of the
>>>> default process manager (hydra)?
>>>>
>>>> -- Pavan
>>>>
>>>> On 06/13/2013 08:05 AM, Pavan Balaji wrote:
>>>>
>>>>> What's "-phrase"? That's not a recognized option. I'm not sure where
>>>>> the /dev/mem check is coming from. Try running ~/main without mpiexec
>>>>> first.
>>>>>
>>>>> -- Pavan
>>>>>
>>>>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote:
>>>>>
>>>>>> Hello all,
>>>>>>
>>>>>> I am trying to use two raspberry-pi to sample and then process some
>>>>>> data. The first process samples while the second processes and vice
>>>>>> versa. To do this I use gpio and also mpich-3.0.4 with the process
>>>>>> manager smpd. I have successfully run cpi on both machines (from the
>>>>>> master machine). I have also managed to run a similar program but
>>>>>> without the MPI, this involved compiling with gcc and when running
>>>>>> putting sudo in front of the binary file.
>>>>>>
>>>>>> When I combine these two processes I get various error messages.
>>>>>> For input:
>>>>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
>>>>>> the error is:
>>>>>> Can't open /dev/mem
>>>>>> Did you forget to use 'sudo .. ?'
>>>>>>
>>>>>> For input:
>>>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
>>>>>> the error is:
>>>>>> sudo: mpiexec: Command not found
>>>>>>
>>>>>> I therefore put mpiexec into /usr/bin
>>>>>>
>>>>>> now for input:
>>>>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
>>>>>> the error is:
>>>>>> Can't open /dev/mem
>>>>>> Did you forget to use 'sudo .. ?'
>>>>>>
>>>>>> Does anyone know how I can work around this?
>>>>>> Thanks,
>>>>>> Eibhlin
>>>>>>
>>>>>>
>>>>>>
>>>>>> ______________________________**_________________
>>>>>> discuss mailing list discuss at mpich.org
>>>>>> To manage subscription options or unsubscribe:
>>>>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>>>>>
>>>>>> --
>>>> Pavan Balaji
>>>> http://www.mcs.anl.gov/~balaji
>>>> ______________________________**_________________
>>>> discuss mailing list discuss at mpich.org
>>>> To manage subscription options or unsubscribe:
>>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>>> ______________________________**_________________
>>>> discuss mailing list discuss at mpich.org
>>>> To manage subscription options or unsubscribe:
>>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>>>
>>> ______________________________**_________________
>>> discuss mailing list discuss at mpich.org
>>> To manage subscription options or unsubscribe:
>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>> ______________________________**_________________
>>> discuss mailing list discuss at mpich.org
>>> To manage subscription options or unsubscribe:
>>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>>
>>
>> ______________________________**_________________
>> discuss mailing list discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>> ______________________________**_________________
>> discuss mailing list discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>> ______________________________**_________________
>> discuss mailing list discuss at mpich.org
>> To manage subscription options or unsubscribe:
>> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>>
>>
> --
> Pavan Balaji
> http://www.mcs.anl.gov/~balaji
> ______________________________**_________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/**mailman/listinfo/discuss<https://lists.mpich.org/mailman/listinfo/discuss>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/c7e368ec/attachment.html>
More information about the discuss
mailing list