[mpich-discuss] Running an mpi program that needs to access /dev/mem
Lee, Eibhlin
eibhlin.lee10 at imperial.ac.uk
Fri Jun 14 11:18:06 CDT 2013
Thank you Antonio.
http://www.skpang.co.uk/blog/archives/615 If you follow the steps to download adc you will get 4 files that need to be included in compilation: gb_common.h gb_spi.h gb_common.c and gb_spi.c I have included the files here as well.
I compile them using
mpicc example_for_MPICH.c gb_common.c gb_spi.c examp -lm -Lbbcm2835-1.25/src -lbcm2835
and execute in the standard manner for smpd but this will be different in hydra. I normally run on 2 machines and have 2 processes started.
If you could please reply with the output in full that would help me. (I do NEED /dev/mem it is directly accessed in gb_common.h)
Thanks,
Eibhlin
________________________________________
From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of Antonio J. Peña [apenya at mcs.anl.gov]
Sent: 14 June 2013 16:10
To: discuss at mpich.org
Subject: Re: [mpich-discuss] Running an mpi program that needs to access /dev/mem
Hi Eibhlin,
If you share a piece of code to test your issue, I can run it for you using
hydra to check if that solves it.
Antonio
On Friday, June 14, 2013 11:27:25 AM Lee, Eibhlin wrote:
> I found that the reason we want to access /dev/mem is to setup memory
> regions to access the peripherals. (We are trying to read the output of an
> ADC). At this point it becomes more a linux/raspberry-pi specific problem
> than an MPICH problem. Although the fact that you can't run a program that
> needs access to memory mapping (even as the root user) seems something that
> MPICH could improve on for future versions. I know I am using smpd instead
> of hydra so this problem may already be solved. But if someone could
> confirm that, it would be really helpful.
> ________________________________________
> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> Lee, Eibhlin [eibhlin.lee10 at imperial.ac.uk] Sent: 14 June 2013 11:20
> To: discuss at mpich.org
> Subject: Re: [mpich-discuss] Running an mpi program that needs to
> access /dev/mem
>
> Gus,
> I tried running cpi, as is included in the installation of MPI, on two
> machines with two processes. The output message confirmed that it had
> started only 1 process instead of 2. Process 0 of 1 is on raspi
> pi is approximately...
>
> Then it just hung. I think this is because the other machine didn't know
> where to output the data?
>
> When I tried running two processes on the one machine using the wrapper you
> suggested the output was the same but doubled. It didn't hang. This
> confirms that every process was started with rank 0.
>
> I'm not entirely sure why /dev/mem is needed. I'm working in a group and
> another member set up io and gpio and it seemed it needed access to
> /dev/mem I am going to do a strace as suggested by Pavan Balaji to see
> where it is used and see if I can somehow work around it.
>
> Thank you for your help.
> Eibhlin
> ________________________________________
> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of Gus
> Correa [gus at ldeo.columbia.edu] Sent: 13 June 2013 21:11
> To: Discuss Mpich
> Subject: Re: [mpich-discuss] Running an mpi program that needs to
> access /dev/mem
>
> Hi Eibhlin
>
> On 06/13/2013 12:59 PM, Lee, Eibhlin wrote:
> > Gus,
> > I believe your first assumption is correct. Unfortunately it just seemed
> > to hang. I think this might be because each one is being made to have the
> > same rank...
> Darn! I was afraid that it might give only rank 0 to all MPI processes.
> So, with the script wrapper the process being launched by mpiexec may
> indeed be sudo,
> not the actual mpi executable (main) :(
> Then it may actually launch a bunch of separate rank 0 replicas of your
> program,
> instead of assigning to them different ranks.
> However, without any output or error message, it is hard to tell.
>
> No output at all?
> No error message, just hangs?
> Have you tried a verbose flag (-v) to mpiexec?
> (Not sure if it exists in MPICH mpiexec, you'd need to check.)
>
> Would you care to try it with another mpi program,
> one that doesn't deal with /dev/mem (a risky business),
> say cpi.c (in the examples directory), or an mpi version of Hello, world,
> just to see if the mpiexec+sudo_script_wrapper works as expected or
> if everybody gets rank 0?
>
> > It may already be obvious but this is the first time I am using Linux. I
> > had tried sudo $(which mpiexec ....) and sudo $(which mpiexec) ... both
> > without success.
> "which mpiexec" will return the path to mpiexec, but won't execute it.
>
> You could try this (with backquotes):
>
> `which mpiexec` -n 2 ~/main
>
> On a side note, make sure the mpiexec you're using matches the
> mpicc/mpif90/MPI library from the MPICH that
> you used to compile the program.
> Often times computers have several flavors of MPI installed, and mixing
> them just doesn't work.
>
> > Is putting the full path to it similar to/is a symlink? (This still
> > doesn't make main have super user privileges though.)
> No, nothing to do with sudo privileges.
>
> This suggestion was just to avoid messing up your /usr/bin,
> which is a directory that despite the somewhat misleading name (/usr,
> for historical reasons I think),
> is supposed to hold system (Linux) programs (that users can use), but
> not user-installed programs.
> Normally things are that are installed in /usr get there via some Linux
> package manager program
> (yum, rpm, apt-get, etc), to keep consistency with libraries, etc.
>
> I belive MPICH would install by default in /usr/local/ (and put mpiexec
> in /usr/local/bin),
> which is kind of a default location for non-system applications.
>
> The full path suggestion would be something like:
> /path/to/where/you/installed/mpiexec -n 2 ~/main
>
> However, this won't solve the other problem w.r.t. sudo and /dev/mem.
>
> You must know what you are doing, but it made me wonder,
> even if your program were sequential, why would you want to mess with
> /dev/mem directly?
> Just curious about it.
>
> Gus Correa
>
> > Eibhlin
> > ________________________________________
> > From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> > Gus Correa [gus at ldeo.columbia.edu] Sent: 13 June 2013 15:37
> > To: Discuss Mpich
> > Subject: Re: [mpich-discuss] Running an mpi program that needs to
> > access /dev/mem
> >
> > Hi Lee
> >
> > How about replacing "~/main" in the mpiexec command line
> > by one-liner script?
> > Say, "sudo_main.sh", something like this:
> >
> > #! /bin/bash
> > sudo ~/main
> >
> > After all, it is "main" that accesses /dev/mem,
> > and needs "sudo" permissions, not mpiexec, right?
> > [Or do the mpiexec-launched processes inherit
> > the "sudo" stuff from mpiexec?]
> >
> > Not related, but, instead of putting mpiexec in /usr/bin,
> > can't you just use the full path to it?
> >
> > I hope this helps,
> > Gus Correa
> >
> > On 06/13/2013 10:09 AM, Lee, Eibhlin wrote:
> >> Pavan,
> >> I had a lot of trouble getting hydra to work without having to enter a
> >> password/passphrase. I saw the option to pass a phrase in the mpich
> >> installers guide. I eventually found that for that command you needed to
> >> use the smpd process manager. That's the only reason I chose smpd over
> >> hydra. As to your other suggestion. I ran ./main and the same error
> >> (Can't open /dev/mem...) appeared. sudo ./main works but of course
> >> without multiple processes. Eibhlin
> >> ________________________________________
> >> From: discuss-bounces at mpich.org [discuss-bounces at mpich.org] on behalf of
> >> Pavan Balaji [balaji at mcs.anl.gov] Sent: 13 June 2013 14:34
> >> To: discuss at mpich.org
> >> Subject: Re: [mpich-discuss] Running an mpi program that needs to access
> >> /dev/mem
> >>
> >> I just saw your older email. Why are you using smpd instead of the
> >> default process manager (hydra)?
> >>
> >> -- Pavan
> >>
> >> On 06/13/2013 08:05 AM, Pavan Balaji wrote:
> >>> What's "-phrase"? That's not a recognized option. I'm not sure where
> >>> the /dev/mem check is coming from. Try running ~/main without mpiexec
> >>> first.
> >>>
> >>> -- Pavan
> >>>
> >>> On 06/13/2013 06:56 AM, Lee, Eibhlin wrote:
> >>>> Hello all,
> >>>>
> >>>> I am trying to use two raspberry-pi to sample and then process some
> >>>> data. The first process samples while the second processes and vice
> >>>> versa. To do this I use gpio and also mpich-3.0.4 with the process
> >>>> manager smpd. I have successfully run cpi on both machines (from the
> >>>> master machine). I have also managed to run a similar program but
> >>>> without the MPI, this involved compiling with gcc and when running
> >>>> putting sudo in front of the binary file.
> >>>>
> >>>> When I combine these two processes I get various error messages.
> >>>> For input:
> >>>> mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> >>>> the error is:
> >>>> Can't open /dev/mem
> >>>> Did you forget to use 'sudo .. ?'
> >>>>
> >>>> For input:
> >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> >>>> the error is:
> >>>> sudo: mpiexec: Command not found
> >>>>
> >>>> I therefore put mpiexec into /usr/bin
> >>>>
> >>>> now for input:
> >>>> sudo mpiexec -phrase cat -machinefile machinefile -n 2 ~/main
> >>>> the error is:
> >>>> Can't open /dev/mem
> >>>> Did you forget to use 'sudo .. ?'
> >>>>
> >>>> Does anyone know how I can work around this?
> >>>> Thanks,
> >>>> Eibhlin
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> discuss mailing list discuss at mpich.org
> >>>> To manage subscription options or unsubscribe:
> >>>> https://lists.mpich.org/mailman/listinfo/discuss
> >>
> >> --
> >> Pavan Balaji
> >> http://www.mcs.anl.gov/~balaji
> >> _______________________________________________
> >> discuss mailing list discuss at mpich.org
> >> To manage subscription options or unsubscribe:
> >> https://lists.mpich.org/mailman/listinfo/discuss
> >> _______________________________________________
> >> discuss mailing list discuss at mpich.org
> >> To manage subscription options or unsubscribe:
> >> https://lists.mpich.org/mailman/listinfo/discuss
> >
> > _______________________________________________
> > discuss mailing list discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
> > _______________________________________________
> > discuss mailing list discuss at mpich.org
> > To manage subscription options or unsubscribe:
> > https://lists.mpich.org/mailman/listinfo/discuss
>
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
> _______________________________________________
> discuss mailing list discuss at mpich.org
> To manage subscription options or unsubscribe:
> https://lists.mpich.org/mailman/listinfo/discuss
_______________________________________________
discuss mailing list discuss at mpich.org
To manage subscription options or unsubscribe:
https://lists.mpich.org/mailman/listinfo/discuss
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gb_common.c
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/2cb12ade/attachment.c>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gb_common.h
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/2cb12ade/attachment.h>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gb_spi.c
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/2cb12ade/attachment-0001.c>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: gb_spi.h
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/2cb12ade/attachment-0001.h>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: example_for_MPICH.c
URL: <http://lists.mpich.org/pipermail/discuss/attachments/20130614/2cb12ade/attachment-0002.c>
More information about the discuss
mailing list