[mpich-discuss] Running an mpi program that needs to access /dev/mem

Lee, Eibhlin eibhlin.lee10 at imperial.ac.uk
Sat Jun 15 12:19:48 CDT 2013 

Jim,
At the time I was checking it would run on one machine so I only changed it on the one machine I was running the program on.

Eibhlin
________________________________
From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of Jim Dinan [james.dinan at gmail.com]
Sent: 15 June 2013 16:03
To: discuss at mpich.org
Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem

Eibhlin,

Did you make those permissions changes on every node where your program runs?  What happens if you run "mpiexec touch /dev/mem"?

 ~Jim.


On Fri, Jun 14, 2013 at 4:43 PM, Lee, Eibhlin <eibhlin.lee10 at imperial.ac.uk<mailto:eibhlin.lee10 at imperial.ac.uk>> wrote:
Pavan,
sorry when I do run mpiexec id the output is
uid=1000(pi) gid=1000(pi) groups=1000(pi),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),105(netdev),999(input)

regardless of whether I'm in root or my usual user. root at raspi or pi at raspi. Is this output what you would expect?

Jim,
I have tried changing the ownership of /dev/mem by
chmod 755 /dev/mem so that the output of ls -l /dev/mem is
crwxr-xr-x 1 root kmem 1, 1 Jan 1 1970 /dev/mem
but I still can't open /dev/mem inside my program. I also tried with the code 777.

I tried adding my user to the kmem group by doing
usermod -a -G kmem pi
but this doesn't fix the problem.


Have I gotten totally confused and pi isn't my user?

Thank you in advance,
Eibhlin
________________________________
From: discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org> [discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org>] on behalf of Jim Dinan [james.dinan at gmail.com<mailto:james.dinan at gmail.com>]
Sent: 14 June 2013 21:31

To: discuss at mpich.org<mailto:discuss at mpich.org>
Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem

I don't know if this has been suggested, but you could also add your user to the kmem group and chmod /dev/mem so that you have the access you need.

 ~Jim.


On Fri, Jun 14, 2013 at 1:24 PM, Pavan Balaji <balaji at mcs.anl.gov<mailto:balaji at mcs.anl.gov>> wrote:

You can run mpich as root.  There's no restriction on that.  You still haven't tried out my suggestion of running "id" to check what user ID you are running your processes as.  My guess is that you are not setting your user ID correctly.

 -- Pavan


On 06/14/2013 06:27 AM, Lee, Eibhlin wrote:
I found that the reason we want to access /dev/mem is to setup memory regions to access the peripherals. (We are trying to read the output of an ADC). At this point it becomes more a linux/raspberry-pi specific problem than an MPICH problem. Although the fact that you can't run a program that needs access to memory mapping (even as the root user) seems something that MPICH could improve on for future versions. I know I am using smpd instead of hydra so this problem may already be solved. But if someone could confirm that, it would be really helpful.
________________________________________
From: discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org> [discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org>] on behalf of Lee, Eibhlin [eibhlin.lee10 at imperial.ac.uk<mailto:eibhlin.lee10 at imperial.ac.uk>]
Sent: 14 June 2013 11:20
To: discuss at mpich.org<mailto:discuss at mpich.org>
Subject: Re: [mpich-discuss] Running an mpi program that        needs   to      access  /dev/mem

Gus,
I tried running cpi, as is included in the installation of MPI, on two machines with two processes. The output message confirmed that it had started only 1 process instead of 2.
Process 0 of 1 is on raspi
pi is approximately...

Then it just hung. I think this is because the other machine didn't know where to output the data?

When I tried running two processes on the one machine using the wrapper you suggested the output was the same but doubled. It didn't hang. This confirms that every process was started with rank 0.

I'm not entirely sure why /dev/mem is needed. I'm working in a group and another member set up io and gpio and it seemed it needed access to /dev/mem I am going to do a strace as suggested by Pavan Balaji to see where it is used and see if I can somehow work around it.

Thank you for your help.
Eibhlin
________________________________________
From: discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org> [discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org>] on behalf of Gus Correa [gus at ldeo.columbia.edu<mailto:gus at ldeo.columbia.edu>]
Sent: 13 June 2013 21:11
To: Discuss Mpich
Subject: Re: [mpich-discuss] Running an mpi program that needs  to      access  /dev/mem

Hi Eibhlin

On 06/13/2013 12:59 PM, Lee, Eibhlin wrote:
Gus,
I believe your first assumption is correct. Unfortunately it just seemed to hang. I think this might be because each one is being made to have the same rank...

Darn!  I was afraid that it might give only rank 0 to all MPI processes.
So, with the script wrapper the process being launched by mpiexec may
indeed be sudo,
not the actual mpi executable (main)  :(
Then it may actually launch a bunch of separate rank 0 replicas of your
program,
instead of assigning to them different ranks.
However, without any output or error message, it is hard to tell.

No output at all?
No error message, just hangs?
Have you tried a verbose flag (-v) to mpiexec?
(Not sure if it exists in MPICH mpiexec, you'd need to check.)

Would you care to try it with another mpi program,
one that doesn't deal with /dev/mem (a risky business),
say cpi.c (in the examples directory), or an mpi version of Hello, world,
just to see if the mpiexec+sudo_script_wrapper works as expected or
if everybody gets rank 0?


It may already be obvious but this is the first time I am using Linux. I had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both without success.

"which mpiexec" will return the path to mpiexec, but won't execute it.

You could try this (with backquotes):

`which mpiexec` -n 2 ~/main

On a side note, make sure the mpiexec you're using matches the
mpicc/mpif90/MPI library from the MPICH that
you used to compile  the program.
Often times computers have several flavors of MPI installed, and mixing
them just doesn't work.

Is putting the full path to it similar to/is a symlink? (This still doesn't make main have super user privileges though.)

No, nothing to do with sudo privileges.

This suggestion was just to avoid messing up your /usr/bin,
which is a directory that despite the somewhat misleading name (/usr,
for historical reasons I think),
is supposed to hold system (Linux) programs (that users can use), but
not user-installed programs.
Normally things are that are installed in /usr get there via some Linux
package manager program
(yum, rpm, apt-get, etc), to keep consistency with libraries, etc.

I belive MPICH would install by default in /usr/local/ (and put mpiexec
in /usr/local/bin),
which is kind of a default location for non-system applications.

The full path suggestion would be something like:
/path/to/where/you/installed/mpiexec -n 2 ~/main

However, this won't solve the other problem w.r.t. sudo and /dev/mem.

You must know what you are doing, but it made me wonder,
even if your program were sequential, why would you want to mess with
/dev/mem directly?
Just curious about it.

Gus Correa



Eibhlin
________________________________________
From: discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org> [discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org>] on behalf of Gus Correa [gus at ldeo.columbia.edu<mailto:gus at ldeo.columbia.edu>]
Sent: 13 June 2013 15:37
To: Discuss Mpich
Subject: Re: [mpich-discuss] Running an mpi program that needs to       access  /dev/mem

Hi Lee

How about replacing "~/main" in the mpiexec command line
by one-liner script?
Say, "sudo_main.sh", something like this:

#! /bin/bash
sudo ~/main

After all, it is "main" that accesses /dev/mem,
and needs "sudo" permissions, not mpiexec, right?
[Or do the mpiexec-launched processes inherit
the "sudo" stuff from mpiexec?]

Not related, but, instead of putting mpiexec in /usr/bin,
can't you just use the full path to it?

I hope this helps,
Gus Correa

On 06/13/2013 10:09 AM, Lee, Eibhlin wrote:
Pavan,
I had a lot of trouble getting hydra to work without having to enter a password/passphrase. I saw the option to pass a phrase in the mpich installers guide. I eventually found that for that command you needed to use the smpd process manager. That's the only reason I chose smpd over hydra.
As to your other suggestion. I ran ./main and the same error (Can't open /dev/mem...) appeared. sudo ./main works but of course without multiple processes.
Eibhlin
________________________________________
From: discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org> [discuss-bounces at mpich.org<mailto:discuss-bounces at mpich.org>] on behalf of Pavan Balaji [balaji at mcs.anl.gov<mailto:balaji at mcs.anl.gov>]
Sent: 13 June 2013 14:34
To: discuss at mpich.org<mailto:discuss at mpich.org>
Subject: Re: [mpich-discuss] Running an mpi program that needs to access        /dev/mem

I just saw your older email.  Why are you using smpd instead of the
default process manager (hydra)?

     -- Pavan

On 06/13/2013 08:05 AM, Pavan Balaji wrote:
What's "-phrase"?  That's not a recognized option.  I'm not sure where
the /dev/mem check is coming from.  Try running ~/main without mpiexec
first.

     -- Pavan

On 06/13/2013 06:56 AM, Lee, Eibhlin wrote:
Hello all,

I am trying to use two raspberry-pi to sample and then process some
data. The first process samples while the second processes and vice
versa. To do this I use gpio and also mpich-3.0.4 with the process
manager smpd. I have successfully run cpi on both machines (from the
master machine). I have also managed to run a similar program but
without the MPI, this involved compiling with gcc and when running
putting sudo in front of the binary file.

When I combine these two processes I get various error messages.
For input:
mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
the error is:
Can't open /dev/mem
Did you forget to use 'sudo .. ?'

For input:
sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
the error is:
sudo: mpiexec: Command not found

I therefore put mpiexec into /usr/bin

now for input:
sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
the error is:
Can't open /dev/mem
Did you forget to use 'sudo .. ?'

Does anyone know how I can work around this?
Thanks,
Eibhlin



_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss

--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss

_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss


--
Pavan Balaji
http://www.mcs.anl.gov/~balaji
_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss


_______________________________________________
discuss mailing list     discuss at mpich.org<mailto:discuss at mpich.org>
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130615/af005f5a/attachment.html>

More information about the discuss mailing list